PaperlayerView exact revision on arXiv

Source-linked AI summary

Talking with Language Models

James Ravi Kirkpatrick, Alexandru Radulescu, Rachel Katharine Sterken

arXiv:2610.09064v1cs.CYcs.AIcs.CLcs.LG

TL;DR

Human–AI text exchanges invite questions about whether LLMs are conversational participants, including whether their outputs refer, promise, or incur responsibility. The paper recasts these exchanges as artifact-mediated production and uptake of candidate texts, arguing that LLMs generate useful text rather than speak. It concludes that conversational appearance is explained by artifact design and user interpretation, while leaving the adaptation of pragmatic-repair accounts for future work.

  • Problem

    Conversation-like LLM exchanges raise questions about whether there is a participant who can refer, promise, and be answerable under conversational norms.

    Source: §1 · p. 3

  • Method

    The paper introduces the artifactual stance, treating LLM interaction as artifact-mediated production and uptake of candidate text rather than speech.

    Source: §1 · p. 3, §3.1 · p. 10

  • Results

    LLM outputs are candidate texts rather than acts of saying, and the paper concludes that LLMs are not speakers or conversational participants.

    Source: §3.1 · p. 10, §7 · p. 21

  • Takeaways & Limitations

    The conversational appearance of LLM interaction is attributed to artifact design and users’ interpretive capacities and tendencies.

    Source: §1 · p. 3, §7 · p. 21

    The account of hearer-side pragmatic repair remains undeveloped; adapting existing approaches to LLMs is left for future work.

    Source: §4.2 · p. 17

Abstract

from arXiv · show

When we interact with large language models (LLMs), are we having a conversation? They are designed to invite us to treat them as intelligent interlocutors who remember, act, and make commitments. But appearances deceive. We introduce the artifactual stance, a framework that reconceives human-AI interaction as artifact-mediated exchanges of candidate texts. LLM outputs are candidate texts optimized for utility, not utterances bearing meaning or force. LLMs are sophisticated text generators, not speakers. Between sessions, nothing runs; between turns, no one remembers. What persists is a configuration and a transcript. The "conversation" is a user's solo performance, interpretive labour disguised by interface and artifact design. This shift dissolves recent philosophical puzzles. Questions about what 'I' and 'you' refer to in AI exchanges, about whether systems can lie or be held to promises, about the identity of our supposed interlocutors all rest on a false presupposition. There is no speaker behind the screen, hence no one to refer to, no one to hold responsible. What feels like dialogue with someone is interaction with an artifact that generates text at unprecedented scale and fit. By abandoning the conversational framing, we see these systems for what they are: immensely sophisticated artifacts that afford varied uses. The philosophical questions that matter are about the normative underpinnings of design, adoption, authorization, and human practices of use.

1 Introduction

Fluent, conversation-like exchanges invite users to treat LLMs as interlocutors, raising questions about reference, memory, promises, and responsibility. The paper instead characterizes them as artifact-mediated exchanges of candidate texts, with users supplying the interpretive and communicative work.

1 Introduction

  • The exchange’s conversational surface suggests a persistent speaker who remembers, addresses Maya, and can promise, but that appearance alone cannot establish anyone is bound by conversational norms.Those descriptions matter because they can carry metaphysical and evidential import, including downstream questions about responsibility.Source: §1 · pp. 2–3
  • Predictive text also generates material that users adopt in human communication, while differences in framing, delivery, and use do not establish a speaker behind the output.Theo is framed as a caring turn-taking partner; autocomplete offers short, locally useful completions for insertion into a human conversation.Source: §1 · p. 2
  • The paper asks whether human–AI text exchange is genuine conversation or only resembles one, a characterization that shapes questions about meaning, reference, system persistence, and responsibility.Its strategy is to characterize the interaction before addressing those downstream questions.Source: §1 · p. 3
  • Human–AI text exchange is artifact-mediated production and uptake of candidate text, not speech itself; its conversational resemblance arises from artifact design and users’ interpretive tendencies.The paper proposes “candidate string” and “candidate exchange” as alternatives to more anthropomorphic descriptions.Source: §1 · p. 3
  • The artifactual account rejects dismissing users as confused while denying that an interlocutor speaks when an LLM produces output.It treats such systems as sophisticated and useful artifacts, while leaving users’ responses intelligible given the interaction’s normative surface.Source: §1 · p. 4
  • The paper develops the artifactual stance and candidate-generator account, then argues that candidate exchanges are not conversations and applies that account to reference and responsibility.It presents the approach as conceptual engineering rather than assuming human-human interaction concepts transfer directly to LLMs.Source: §1 · pp. 3–4

2 AI Systems Are Artifacts, Not Intentional Agents

The paper proposes the artifactual stance as an alternative to treating LLMs as intentional agents: explain them through design, affordances, and users’ practices. It applies that stance through comparisons with familiar artifacts and interaction systems, arguing that LLMs’ greater sophistication changes scale and fit, not the basic kind of interaction.

2.1 Dennett’s Interpretational Stances

  • Attributing beliefs and desires is warranted only when it improves explanation or prediction; the authors argue that artifact design can explain LLM behavior without settling whether outputs are speech acts or exchanges are conversations.Even if intentional descriptions help predict behavior, predictive utility alone does not resolve those constitutive questions.Source: §2.1 · p. 5, §2.2 · p. 6

2.2 The Artifactual Stance

  • The artifactual stance explains LLM behavior through artifacts’ design, outputs, affordances, and users’ uses, without treating them as full cognitive agents or mere “stochastic parrots.”It permits open-ended uses and opaque mechanisms; explanations may appeal to optimization targets, training history, and affordance profiles.Source: §2.1 · p. 6, §2.2 · p. 7
  • The artifactual stance treats prediction through design as useful when it suffices, while allowing artifacts to have open-ended uses and mechanisms their designers or users do not understand.The design stance is a special subtype of the artifactual stance, where design knowledge suffices to predict behavior.Source: §2.2 · pp. 6–7
  • Affordances relate artifact features to users’ capacities and uses, while LLMs’ open-ended interfaces leave users to work out what the systems can do and how to elicit it.That interpretive labor falls on users, and the paper treats user practices as relevant to explaining artifact use.Source: §2.2 · pp. 7–8

2.3 Using the Artifactual Stance

  • Greeting cards, answering machines, and autocomplete separate artifact-produced or stored text from the user’s deployment of it; autocomplete leaves selection and endorsement with the user.The cases illustrate linguistically mediated interaction that need not be conversation between minded or norm-governed participants.Source: §2.3 · pp. 8–9
  • Modern LLMs are a jump in scale, fit, and sophistication from ELIZA, but not a change in nature; their fine-tuned affordances help explain their conversational appearance.The authors attribute that appearance to successful design and a small but noticeable failure rate.Source: §2.3 · p. 9

3 Human–AI Interaction Under the Artifactual Stance: Candidate Texts and Candidate Exchanges

The artifactual stance treats LLMs as designed artifacts that generate useful candidate texts, which users interpret and put to use. Candidate exchanges divide production, interpretation, and commitment differently from conversation, while their design helps explain both their usefulness and conversational feel.

3 Human–AI Interaction Under the Artifactual Stance: Candidate Texts and Candidate Exchanges

  • The artifactual stance asks what LLMs are designed to produce, what their outputs afford, and how users deploy them.It frames human–AI interaction as artifact-mediated production and uptake rather than assuming human–human concepts transfer straightforwardly.Source: §3 · p. 9

3.1 Candidate Texts

  • LLM outputs are candidate strings—token sequences shaped as text and optimized for utility, not necessarily truth, belief, sincerity, or endorsement.Their contextual and structural interpretations, when fixed, are fixed at reception rather than generation.Source: §3.1 · p. 10
  • Fluency, relevance, and apparent attentiveness are engineered affordances arising from design, training, and fine-tuning, not evidence that an interlocutor spoke.The authors locate responsibility for reliably inviting users to construct an interlocutor in design rather than speakerhood.Source: §3.1 · p. 10, §3.2 · p. 10

3.2 Candidate Exchanges

  • Candidate exchanges cycle through user prompting, artifact generation, user reception, and user uptake; the user frames, interprets, and commits, if at all.Uptake can include believing, revising, sending, acting on, or ignoring the candidate, and answering it can start the next cycle.Source: §3.2 · p. 11
  • This division of labor makes exchanges useful and conversational-seeming: the artifact supplies linguistic products of recall and synthesis, while users interpret, endorse, and act.Users may adopt text receptively as content or appropriatively as their own speech, becoming answerable for the resulting utterance.Source: §3.2 · pp. 11–12

4 Human–AI Interactions Are Not Conversations

The paper argues that LLM generation produces candidate text, not utterances, and that human–LLM exchanges therefore lack the constitutive structure of conversation. Users perform the interpretive and commitment-bearing work, while no determinate system entity persists as a promiser.

4.1 Generation is Not Utterance

  • Generation supplies no context of utterance, so it produces neither an event of saying nor utterance content; users may instead take up the text as saying something.Even inherited non-indexical content cannot give generated ‘I’ a semantic value without a context fixing a speaker.Source: §4.1 · pp. 12, 13, 15
  • Generation selects no principled speaker, addressee, or production event to fix indexicals; choosing among token emission, display, or other events merely stipulates parameters.The resulting ‘I’ lacks a fixed referent, and ‘tomorrow’ can be fixed before a user reads the text.Source: §4.1 · pp. 13–14
  • Prompts and transcripts condition which string is useful, but do not fix indexical meanings as a speaker-occupied context of utterance would.Reception-side context fixing assigns the relevant communicative and cognitive work to the human user.Source: §4.1 · pp. 13–14
  • Regenerating, editing, or discarding outputs is not treated as retracting, revising, or abandoning an LLM speech act, supporting the candidate-text account.The authors argue that treating such outputs as utterances would require attributing widespread conceptual error to users.Source: §4.1 · p. 15

4.2 Human–LLM Interactions are Not Conversations

  • Because conversation requires illocutionary contributions and generation issues no locutionary act, an exchange with contributions from only the human is not a conversation.Conversation is characterized as a joint activity constituted by speech acts.Source: §4.2 · pp. 15–16
  • Transcript feedback provides causal continuity, not deontic continuity; defining a persistent interlocutor as commitment-bearer instead presupposes that the exchange is conversation.Quasi-commitments may be attributed by users, but cannot be enforced, retracted, or held against anyone; context-window overflow also drops earlier commitments without marking them lost.Source: §4.2 · pp. 16–17
  • The conversational resemblance is explained by one participant performing both parts with an artifact, through user-side pragmatic interpretation that the authors leave for future work.The relevant repair mechanisms need adaptation because they often rely on attributions of intentions and other mental states.Source: §4.2 · p. 17

5 Reference Without Speakers

Names and pronouns have no uniform role in human–AI interaction: their meaning depends on whether users prompt, interpret, or appropriate candidate text.

5.1 Prompting Uses

  • Prompting names can steer generation or configure a role, while a name may also refer thinly to an artifact without making it a thick interlocutor.In prompts, “Theo” helps signal a mode of generation; such use does not by itself pick out a minded conversational partner.Source: §5.1 · pp. 17–18

5.2 Receptive Uses

  • At generation, candidate pronouns need not refer; users may later supply constructed referents through reception, while trained patterns explain autobiographical-sounding text.Even stored and reprocessed dialogue does not by itself give the system an autobiographical perspective.Source: §5.2 · pp. 18–19

5.3 Appropriative Uses

  • When users appropriate candidate text as their own speech, its indexicals take ordinary meanings in the user's context, without making the system the original speaker.The user resolves “I,” “you,” and “now” for the intended message; drafting and speakerhood can come apart.Source: §5.3 · p. 19

6 Consequences for AI Reference and Responsibility

Reference puzzles arise because names can shift among artifacts, records, and user constructions, while responsibility depends on institutional and normative arrangements rather than speakerhood.

6 Consequences for AI Reference and Responsibility

  • A name such as “Theo” can refer to a service, model, instance, transcript, or user-constructed entity, and the appropriate individuation varies with context and purpose.Treating these associated objects as one helps dissolve puzzles about AI reference.Source: §6 · p. 19
  • A persistent transcript can create an appearance of a persisting interlocutor by storing, retrieving, and reprocessing text without requiring a remembering subject.The transcript supplies continuity across cycles and exchanges.Source: §6 · pp. 19–20
  • User uptake can construct a character-like source, and conflating it with the product and transcript creates an impression of one interlocutor that token generation alone does not establish.The computational entity causally responsible for tokens is not thereby speaking.Source: §6 · p. 20
  • Responsibility for generated text depends on who produced, authorized, adopted, or deployed it; Air Canada was held responsible for inaccurate chatbot information without settling chatbot speakerhood.The correspondence example likewise locates answerability with the organization that authorized and sent the letters.Source: §6 · p. 20
  • The argument offers an alternative picture, not a categorical proof that conversation requires normative structure or that artificial systems can never be interlocutors.Its methodological caution is not to assume human-human interaction categories apply neatly to human–AI exchange.Source: §6 · p. 20

7 Concluding Remarks

The paper reframes current LLM interaction as artifact-mediated exchange of candidate text rather than conversation. It argues that conversational appearance alone does not establish speakerhood, while the approach also extends to candidate actions adopted by users.

7 Concluding Remarks

  • The artifactual stance treats LLMs as artifacts that generate candidate text for varied uses, not as speakers producing utterances or participants in conversations.Conversational appearance follows from artifact design and user uptake, rather than establishing that the artifact is a participant.Source: §7 · p. 21
  • Reference and answerability are separated: names and pronouns work differently across uses, while responsibility follows normative and institutional arrangements governing text.Those arrangements include production, authorization, adoption, use, and deployment.Source: §7 · p. 21
  • Fluent, coherent, or convincingly conversational output supports taking an artifact's affordances seriously but does not by itself show that its outputs are utterances.The paper does not claim artificial conversation is impossible; current systems' conversational status should not be inferred from appearance alone.Source: §7 · p. 21
  • The candidate-and-uptake account also applies beyond text: an AI-generated candidate action becomes attributable to an agent when that agent adopts it, including through pre-authorization.A pre-authorized candidate booking can result in the user's booking a table; differences in adoption across speech and other actions remain for future work.Source: §7 · p. 21
  • The methodological lesson is to identify the kind of human–AI interaction before applying concepts such as believing, saying, promising, remembering, or referring.The paper cautions against importing human-human concepts merely because technology makes them seem natural.Source: §7 · p. 21
Loading 2610.09064v1…