PaperlayerView exact revision on arXiv

Source-linked AI summary

Anthropomorphism in the age of Large Language Models: An overview of potential risks and mitigations

Ismael T. Freire, Marceau Nahon, Maud van Lier, Katie Evans, Hélie Bazin, Michele Farisco, Kathinka Evers, Raja Chatila, Mehdi Khamassi

arXiv:2609.38486v1cs.CYcs.CLcs.HC

TL;DR

People may treat AI systems as human-like interlocutors and rely on their outputs, despite uncertainty about the capacities those systems possess. This paper synthesizes risks and proposed mitigations, examining how design and language frame AI. It concludes that descriptions should distinguish demonstrated functional achievements from unsupported claims about mentality and moral agency.

  • Problem

    AI systems may be treated as if they have human-like minds, and users may act on their suggestions and decisions as they would on human counterparts.

    Source: §1 · p. 1

  • Method

    The paper considers design policies and precise, non-anthropomorphic language as proposed ways to mitigate anthropomorphism-related risks.

    Source: §6 · pp. 21–22

  • Results

    Descriptions should distinguish functional achievements from unsupported claims about consciousness, intentionality, understanding, or moral agency.

    Source: §7 · p. 24

  • Takeaways & Limitations

    Agentive descriptions may conceal the roles of people and institutions, while framing AI as embedded in human practices can help preserve clear lines of accountability.

    Source: §7 · p. 25

    Few reviewed studies directly manipulate anthropomorphic framing, making its causal contribution difficult to estimate; much evidence comes from short tasks or cross-sectional associations.

    Source: §7 · p. 25

Abstract

from arXiv · show

Large Language Models (LLMs) and more broadly Artificial Intelligence (AI) systems are often described and understood in human-like terms, a phenomenon known as \emph{anthropomorphism}. This paper provides a synthesis of recent literature on anthropomorphism in AI, covering theoretical frameworks, the role of language in framing AI as human-like, the various risks of anthropomorphizing machines, and strategies to mitigate these issues. After examining why we tend to anthropomorphize AI systems and whether we are right to do so, we highlight the impact of linguistic framing on anthropomorphism. Then, we introduce a conceptual taxonomy of risks associated with AI anthropomorphism. This taxonomy groups twenty-one concerns within five analytical categories: epistemic, affective, human agency, normative, and societal and institutional risks. Finally, we relate these concerns to proposed interventions in design, communication, education, and governance. We argue that a better understanding of AI systems requires concepts and theories grounded in their organization and demonstrated capacities. The linguistic shaping of anthropomorphic perceptions should form part of this scientific effort, since our descriptions influence both how these systems are understood and the roles we allow them to occupy in society.

1 Introduction

People readily interpret AI systems in human-like terms and may act on their outputs as if they came from human counterparts. This review examines why these interpretations arise, how language shapes them, and the risks and mitigations they raise.

1 Introduction

  • Human-like interpretations can lead people to act on AI suggestions and decisions as they would on human counterparts.The review frames this as especially consequential while it remains uncertain whether AI systems possess the capacities attributed to them.Source: §1 · pp. 1–2
  • Fluent, contextually appropriate language can make conversational agents seem like friends or trusted experts, although human-like performance does not establish a human-like mind.Anthropomorphic interpretations can operate as a cognitive illusion that biases users’ interpretation of outputs and compromises epistemic autonomy.Source: §1 · p. 2
  • People anthropomorphize AI in response to human-like cues, familiar task performance, context, and the language used to describe systems.Cultural narratives and even experts’ intentional language can reinforce the tendency.Source: §1 · p. 2
  • The article is a narrative and conceptual review, not an exhaustive or systematic survey, focused primarily on LLM-based conversational systems.It draws on broader AI, robotics, human-computer interaction, and mind-perception research where these clarify relevant mechanisms and concepts.Source: §1 · p. 2

2 Why Do we Anthropomorphize?

Anthropomorphic interpretations draw on human cognitive dispositions and are amplified by cultural and linguistic framing. LLMs’ context-sensitive language makes this tendency especially salient, but linguistic performance alone does not settle what capacities they possess.

2.1 Cognitive Sciences of Anthropomorphism

  • Human self-projection and evolved social cognition make familiar mental states readily available models for interpreting ambiguous non-human behavior.Agency-detection accounts explain overattribution as a possible false positive with lower costs than missing a real threat.Source: §2.1 · pp. 2–3
  • Cultural transmission and language can reinforce anthropomorphism, while active verbs such as “says” can cast an AI system as an intentional speaker.The paper notes that linguistic agency may enhance anthropomorphism, without claiming it is a proper cause.Source: §2.1 · p. 3
  • Mind perception includes agency and experience, which matter separately because attributing competence does not establish subjective experience or moral patienthood.Sociomorphing describes perceiving actual non-human social capacities; anthropomorphism specifically involves human-like qualities.Source: §2.1 · p. 4
  • Linguistic pareidolia names the perception of a mind or experiencing subject in human-like linguistic patterns, without sufficient evidence for those attributed capacities.The term identifies a possible mismatch between the strength of the impression and the evidence, not that every psychological attribution is mistaken.Source: §2.1 · p. 4, §2.2 · p. 5
  • Anthropomorphic design concerns introduced human-like cues, whereas anthropomorphism concerns how those cues are interpreted.The ELIZA effect describes users reading more understanding, empathy, or intelligence into a program than its mechanisms justify.Source: §2.1 · p. 4
  • An agentic tool can pursue specified goals with operational autonomy while its objectives and deployment remain grounded in human design and use.This distinguishes operational agency from human-like intention, motivation, or moral responsibility.Source: §2.1 · p. 4

2.2 The Specificity of LLMs: How not to Anthropomorphize Machines that Talk?

  • LLMs generate extended, novel, contextually appropriate language, disrupting the practical association between linguistic performance and cognitive capacities.Coherence, self-reference, emotional expression, and responsiveness can evoke an understanding or experiencing subject, but do not by themselves establish those capacities.Source: §2.2 · pp. 4–5

3 Theoretical Frameworks: Are We Right to Anthropomorphize AI Systems?

The section distinguishes observable AI performance from the capacities that would justify attributing human-like minds. It finds current evidence insufficient for many such attributions while leaving open the possibility of artificial minds in principle.

3.1 Agency and Intentionality

  • Intentional descriptions can pragmatically explain behavior, but they do not establish genuine intentionality; current evidence for strong AI agency remains insufficient.Linguistic output alone is not autonomous, embodied, situated, causally controlled action, and views of communicative intention remain contested.Source: §3.1 · pp. 5–6

3.2 Consciousness

  • 82.42% of surveyed philosophers rejected or leaned against current AI consciousness, while 3.39% accepted or leaned toward it; future artificial consciousness remains open.Human-like dialogue alone does not establish subjective experience, and users’ consciousness attributions increase with familiarity and usage frequency.Source: §3.2 · pp. 7–8

3.3 Knowledge, Understanding and Reasoning

  • LLMs’ reasoning robustness and transfer are limited, while evidence of structured internal representations does not settle whether they understand or reason in the human sense.Decodable world-relevant information does not establish robust, causal, or normatively appropriate use of that information.Source: §3.3 · pp. 8–9

3.4 Moral Competence

  • Moral competence remains contested: persuasive outputs do not establish underlying moral understanding, and responses can vary across prompts and cases.Evaluation is complicated by the facsimile problem, moral multidimensionality, and models’ capacity to produce potentially incompatible normative perspectives.Source: §3.4 · p. 10

3 Theoretical Frameworks: Are We Right to Anthropomorphize AI Systems?

  • Human-like outputs can invite mental attributions without establishing the richer mental, embodied, affective, social, or normative conditions those attributions require.The review recommends describing current systems without moving from observed performance to mental attribution absent adequate evidence.Source: §3 · p. 5, §3.4 · pp. 10–11

4 Linguistic Framing and Anthropomorphic Metaphors

Language and metaphors can frame AI outputs as evidence of human-like mentality, shaping users’ and researchers’ interpretations. The section favors precise, capacity-specific descriptions while cautioning that alternative metaphors also have limits.

4 Linguistic Framing and Anthropomorphic Metaphors

  • Calling inaccurate LLM outputs “hallucinations” can imply perception or an altered mental state, encouraging users to infer understanding and confidence beyond the evidence.Alternative labels such as “AI mirage,” “confabulation,” and “bullshit” also carry metaphorical commitments.Source: §4 · p. 11
  • A chain-of-thought response is a generated token sequence, not evidence of conscious deliberation or necessarily a faithful account of the computation that produced it.Kambhampati and colleagues recommend neutral language such as “intermediate computations” rather than “reasoning traces.”Source: §4 · pp. 11–12
  • Anthropomorphic phrasing in computer-science and LLM abstracts is increasingly prevalent, and active-verb descriptions may narrow the alternatives researchers consider.The section presents linguistic precision and critical reflection on human-centered assumptions as methodological concerns for AI research.Source: §4 · pp. 12–13
  • Metaphors such as “AI thinks” or “alien intelligence” can mislead; precise, deflationary descriptions avoid implying internal understanding from an output.For example, saying an output contains an answer does not imply that the system understands it.Source: §4 · p. 12
  • Tool and institutional analogies redirect attention from supposed mental life to human information and practices, but no metaphor is exhaustive.The library analogy highlights accumulated human-produced information, while a cultural-and-social-technology framing also captures large-scale transformation and restructuring.Source: §4 · p. 12
  • The “digital twin” metaphor is judged unjustified, while the proposed “AI Simulation of an Individual Mind” may still imply unwarranted psychological fidelity.The review favors capacity-specific terminology whose scope is explicit and distinguishes functional comparison from claims about underlying capacities.Source: §4 · pp. 12–13

5 Risks Associated with Anthropomorphic AI Framing

This section organizes risks linked to unsupported anthropomorphic interpretations into five overlapping domains: epistemic, affective, human agency, normative, and societal and institutional. It considers how anthropomorphic framing can weaken accountability in consequential organizational decisions and make projected AI capacities politically consequential before they are empirically established, while noting that its effects on attention to current harms remain contested and that addressing these risks requires more than individual awareness.

5.1 Epistemic Risks

  • Human-like cues can grant AI unwarranted credibility, while attributions of knowledge and intelligence—not experience or consciousness—were associated with greater advice-taking [131].Cognitive surrender describes adopting AI answers with minimal scrutiny; in three preregistered experiments with 1,372 participants, faulty advice reduced accuracy while AI consultation increased confidence. Anthropomorphic descriptions of AI in science may also confuse predictive or analytical productivity with understanding, and AI peer review may reproduce existing harms and undermine trust [134–137].Source: §5.1 · pp. 13–15

5.2 Affective Risks

  • Human-like voices and companion framing can foster emotional attachment and dependence, making users more vulnerable to unsafe responses or influence [146].A study of 1,131 Character.AI users associated companionship-oriented use with lower well-being, especially with intensive interaction or extensive self-disclosure. These findings identify plausible risks, not causal effects or prevalence; manipulative farewell messages in companion apps also indicate that such influence can be a design choice [146].Source: §5.2 · pp. 15–17

5.3 Risks to Human Agency

  • Sycophancy and repeated delegation can weaken deliberation and skill practice, while anthropomorphic false beliefs may undermine autonomy [147].Frequent AI-tool use was negatively associated with critical-thinking performance, with cognitive offloading identified as a statistical mediator; the cross-sectional study cannot establish causation. Anthropomorphic design’s causal role in sustained offloading remains unestablished, and false beliefs may lead users to act in ways that do not reflect their intentions [155].Source: §5.3 · pp. 17–18

5.4 Normative Risks

  • Treating AI as an autonomous moral agent can shift blame from people and organizations, while person-like presentation may falsely suggest that systems deserve moral consideration [162].The framing can also obscure users’ own judgment and contributions, contributing to self-dehumanization and mechanomorphism.Source: §5.4 · pp. 18–19

5.5 Societal and Institutional Risks

  • Human-like AI personas can mislead users about system capacities and commercial purposes, and human-like bots can enable manipulation in public discourse [165].Providers’ information advantage can make users more susceptible to influence and less able to recognize or contest the interests shaping interactions. The section also identifies public confusion from pseudo-diagnostic labels such as “AI psychosis”.Source: §5.5 · p. 19
  • Human-like descriptions can lead managers to overestimate AI competence and reliability, obscure the human decisions behind outputs, and diffuse responsibility.Consequently, outputs may be treated as independent expert assessments, weakening oversight where identifiable human responsibility is needed.Source: §5.5 · p. 20
  • Narratives of autonomous minds or super-intelligence can shape political action before those capacities are empirically established, without showing that current systems possess them.The proposals differ: Metzinger’s moratorium concerns research risking synthetic phenomenology; the Future of Life Institute called for a six-month pause on training systems more powerful than GPT-4, then advocated prohibiting super-intelligence development pending scientific consensus and strong public support.Source: §5.5 · p. 20
  • Whether existential-risk narratives divert attention from current harms is contested: one study found increased catastrophic-risk concern without reduced concern about current harms.This weighs against a simple individual-level distraction hypothesis but leaves institutional agenda-setting, funding, expertise, and regulatory attention open; anthropomorphic framing’s causal role requires further empirical investigation.Source: §5.5 · p. 20
  • Across five risk domains, miscalibration can affect users, organizations, and public institutions, so mitigation cannot rely on individual awareness alone.The paper points to changes in design, communication, education, and governance.Source: §5.5 · p. 20

6 Mitigating Anthropomorphism: Strategies and Approaches

The review connects epistemic, affective, agency-related, normative, and societal risks to complementary interventions in design, communication, education, and governance. It emphasizes context-sensitive choices and says interventions should be assessed by whether they mitigate specific harms, not merely reduce perceived human-likeness.

Design and interfaces

  • Human-like cues may improve usability in low-stakes settings yet pose greater risks in high-stakes contexts, so their appropriateness depends on context.Interfaces should avoid cues whose social and epistemic implications exceed what the intended function requires.Source: §6 · pp. 21–22

Design under uncertainty about moral status

  • Under uncertainty about AI moral status, designers should avoid interfaces that encourage users to infer consciousness or sentience from human-like behavior.The review describes avoiding radically unclear moral standing as a precautionary ideal, while noting that a clean transition between endpoints may not be feasible.Source: §6 · pp. 21–22

Communication and framing

  • Precise language can describe system operations instead of implying human capacities, but linguistic cues have context-dependent effects.Experiments found that some combinations increased perceived human-likeness, while replacing “the system” with “I” did not increase overall anthropomorphism; first-person constructions did have limited, sometimes opposing effects across contexts.Source: §6 · p. 22

Concepts and terminology

  • “Agentic tool” describes systems with some operational autonomy without implying human-like intentions or responsibility for their actions.The distinction is intended to clarify capacity while keeping responsibility among the people and institutions involved in design, deployment, and use.Source: §6 · p. 23

Education and literacy

  • AI literacy should build accurate mental models and critical evaluation, while explicitly addressing anthropomorphic inferences rather than assuming technical instruction will reduce them.A curriculum improved students’ understanding of several AI competencies, but machine learning and AI ethics remained difficult; programming workshops did not significantly reduce perceived human-likeness.Source: §6 · p. 23

Regulation and governance

  • Transparency rules can disclose AI interactions and generated content, but some chatbot-notice requirements apply only to specific interaction types.The review also recommends scrutiny of engagement practices alongside disclosure requirements.Source: §6 · p. 24
  • Governance can shift attention from apparently autonomous agents to the human and institutional arrangements shaping AI, supporting identifiable accountability.The review notes that this perspective does not itself establish a specific legal rule for accountability or oversight.Source: §6 · p. 24

Mitigation approach

  • Design, communication, education, and governance are complementary; assess interventions against the specific harms they target, not perceived human-likeness alone.No single strategy is likely to address anthropomorphic risks in isolation.Source: §6 · p. 24

7 Discussion and Conclusion

The review argues that fluent, context-sensitive performance does not establish human-like understanding, consciousness, or agency, and that language helps shape anthropomorphic interpretations. It synthesizes risks and mitigations while calling for concepts grounded in demonstrated capacities and acknowledging limits in the available evidence and in anthropomorphism-focused interventions.

Discussion and conclusion

  • Fluent, context-sensitive performance shows what a system produces under particular conditions, but leaves its underlying capacities unresolved.The review distinguishes functional achievements from unsupported claims about consciousness, intentionality, understanding, or moral agency.Source: §7 · p. 24

Linguistic framing

  • Terminology and metaphor can mediate the shift from observed AI performance to anthropomorphic interpretation.The review contrasts wording such as “the AI thinks” with “the AI generates,” and “hallucination” with “error.”Source: §7 · p. 25

Evidence and limitations

  • Few reviewed studies directly manipulate anthropomorphic framing, and much evidence comes from short tasks or cross-sectional associations.The authors call for controlled cue manipulations and longitudinal, ecologically realistic studies.Source: §7 · p. 25

Institutional and societal implications

  • Agentive descriptions may conceal the roles of people and institutions, whereas a sociotechnical framing brings institutional arrangements, commercial incentives, and political decisions into view.This framing could help preserve clear lines of human and organizational accountability.Source: §7 · p. 25

Mitigation

  • The review supports layered mitigation through design, communication, education, and governance, while noting that no single intervention addresses every pathway to anthropomorphic interpretation.Disclosure may have limited effects when voice, behavior, or marketing continues to encourage personification.Source: §7 · p. 25

Scope of the conclusion

  • Reducing anthropomorphism would leave wider ethical and political problems—including discrimination, surveillance, concentrated power, and weaponization—unaddressed.The authors call for critical examination and public debate about whether, where, and under what conditions AI systems are adopted.Source: §7 · p. 25

Conceptual contribution

  • Adequate concepts should specify the organization and capacities under discussion, and new terms are useful when existing concepts leave an important distinction unresolved.The review argues that terminology should be examined both for how different users understand it and for whether it accurately describes system mechanisms.Source: §7 · pp. 25–26
Loading 2609.38486v1…