PaperlayerView exact revision on arXiv

Source-linked AI summary

Where is the Mind? Persona Vectors and LLM Individuation

Pierre Beckmann, Patrick Butlin

arXiv:2604.17031v3cs.CLcs.AI

TL;DR

The paper asks which entities associated with LLMs, if any, should count as minds. It uses mechanistic interpretability and empirical persona research to assess candidate views, arguing for the virtual instance and proposing two persona-based alternatives. It concludes that attention streams support psychological connections across token-time, while persona regions could ground additional individuation views if confirmed.

  • Problem

    The paper addresses which LLM processes and outputs should be attributed to the same minds.

    Source: §1.1 · p. 3

  • Method

    The authors use mechanistic arguments about attention streams and organize empirical work on personas, persona vectors, and persona space around hypotheses about their implementation.

    Source: Introduction · p. 2, §2.1 · p. 6, §3 · p. 11

  • Results

    Attention streams carry mental-state-like representations across token-time, and persona research suggests that stable regions in a low-dimensional persona space could matter for individuation if confirmed.

    Source: Conclusion · p. 26

  • Takeaways & Limitations

    The authors favor the virtual instance view and propose instance-persona and model-persona views that use persona regions to distinguish or unify minds.

    Source: Conclusion · p. 26

    Evidence for a low-dimensional persona space remains limited to one study of smaller open-source models and 275 roles, while the model-persona view lacks psychological connections between instances.

    Source: §3.2 · p. 16, §4.3 · p. 25

Abstract

from arXiv · show

The individuation problem for large language models asks which entities associated with them, if any, should be identified as minds. We approach this problem through mechanistic interpretability, engaging in particular with recent empirical work on persona vectors, persona space, and emergent misalignment. We argue that three views are the strongest candidates: the virtual instance view and two new views we introduce, the (virtual) instance-persona view and the model-persona view. First, we argue for the virtual instance view on the grounds that attention streams sustain quasi-psychological connections across token-time. Then we present the persona literature, organised around three hypotheses about the internal structure underlying personas in LLMs, and show that the two persona-based views are promising alternatives.

Introduction

The paper asks which entities associated with LLMs, if any, should count as minds, a question relevant to identifying potential moral patients. It clarifies and mechanistically supports the virtual instance view, then argues that persona research motivates two additional individuation views.

Introduction

  • The individuation question is whether the model, a physical instance, a virtual instance, or another entity is the subject of any mind-like states attributed to LLMs.The question matters for identifying which entities could be moral patients in discussions of AI welfare.Source: Introduction · pp. 1–2
  • The paper clarifies and reinforces the virtual instance view by appealing to mechanisms of LLM processing.A virtual instance is the model participating in a single conversation, regardless of which hardware realizes its computational steps.Source: Introduction · p. 2
  • The authors argue that LLM individuation literature underemphasizes context-appropriate personas, which LLMs adopt partly for next-token prediction.The simulator framework treats models as role-playing many characters, while newer accounts emphasize the helpful-assistant persona shaped by post-training and system prompts.Source: Introduction · p. 2
  • Persona-vector research and emergent misalignment motivate the authors’ second contribution: presenting empirical work and arguing that personas matter for LLM individuation.Finetuning on a narrow task such as insecure code can push models toward a broadly coherent ‘evil’ persona that generalizes across contexts.Source: Introduction · p. 2
  • The paper presents the virtual instance as its strongest existing candidate and introduces the instance-persona and model-persona views as promising alternatives.It organizes persona research around three hypotheses about persona implementation and identifies open empirical questions for mechanistic interpretability.Source: Introduction · p. 2

1 The problem and proposed solutions

LLM individuation asks which processes and outputs should be attributed to the same minds, across both conversations and time within a conversation. The section surveys candidate entities and their trade-offs, favoring the virtual instance while noting unresolved challenges.

1.1 The individuation problem

  • The individuation problem asks which LLM processes and outputs belong to the same minds, across conversations and across token-time within a conversation.Its synchronic aspect concerns distributed activity; its diachronic aspect concerns persistence despite possible radical behavioral changes.Source: §1.1 · p. 3
  • Proposed desiderata include interactivity, persistence, coherence, faithfulness, unity, and predictive accuracy without slicing the system too finely.For potential moral patients, coherent desires or values may matter because coherent interests appear necessary for being benefited or harmed; consciousness is not the only reason individuation may matter.Source: §1.1 · pp. 3–4
  • Given the varied questions and differences between LLMs and humans, no single individuation solution may prove clearly correct.LLMs may host multiple minds, potentially of multiple kinds.Source: §1.1 · p. 4

1.2 Models, instances and threads

  • The model view identifies an individual with an abstract function defined by an architecture and weight matrix, but its behavior may resist coherent characterization across contexts.The cited objections are that models need not be instantiated, do not change over time, and can behave very differently across contexts.Source: §1.2 · p. 4
  • The physical instance view identifies an individual with particular hardware running a model, but distributed processing and multi-tenancy complicate that mapping.A single conversation may span hardware, while one GPU or server may process multiple conversations in parallel.Source: §1.2 · pp. 4–5
  • The virtual instance view identifies an individual with a model running on one conversation regardless of physical substrate, while the thread view unifies successive virtual instances that inherit conversational context.The authors argue attention heads carry feature-organized information across token-time, providing richer connections than the transcript and supporting the virtual instance view even with model change.Source: §1.2 · p. 5
  • Birch argues that transcript and abstract weight similarity provide too little psychological connection between successive forward passes to ground a persisting entity.He therefore describes the conversational character as a “persisting interlocutor illusion”; the authors counter that attention heads carry forward feature-organized information.Source: §1.2 · p. 5

2 A mechanistic defense of the virtual instance view

The mechanistic account treats attention streams as links that carry feature-organized information from earlier token positions into the current token, sustaining quasi-psychological connections across token-time. It argues that this supports the virtual instance view: model changes replace mental-state-like representations, while server changes using the same model can reconstruct them.

2.1 Attention streams in LLMs

  • Attention layers retrieve cached key-value information from relevant earlier positions into the current token’s residual stream, while the residual stream builds its interpretation and next-token prediction.Keys index information, values encode it, and queries are matched against cached keys to retrieve relevant values.Source: §2.1 · pp. 6–7
  • Planning features can persist through attention streams and bias successive predictions toward a represented outcome, functioning as intention-like states during generation.A feature for “rabbit” activates before the second line is written and is carried forward as generation proceeds.Source: §2.1 · pp. 8–9

2.2 Model change and favouring the virtual instance view

  • When a conversation switches models, the new model rebuilds its own KV cache by pre-filling the transcript, replacing the previous model’s encodings with representations shaped by different weights.The new representations may have similar or different content, but the encodings themselves are replaced.Source: §2.2 · p. 9
  • Because model changes replace mental-state-like representations and often replace their content, the authors argue that a conversation with multiple models hosts successive minds rather than one thread agent.The authors qualify this conclusion as especially suggestive on a representationalist assumption.Source: §2.2 · p. 9

2.3 Model serving and virtual instance continuity

  • When the same model moves servers, pre-filling recreates essentially the same activation pattern; the authors say this preserves representations rather than replacing them.Pre-filling rebuilds the KV cache from the transcript, and the rebuilt cache is extremely similar to the original.Source: §2.3 · p. 10
  • Server migration complicates continuity without undermining the virtual instance view: KV-cache transfer is uninterrupted, whereas pre-filling mediates continuity through the transcript.Pre-filling is near-deterministic, with minor numerical drift that is plausibly too small to change feature content such as the rabbit intention.Source: §2.3 · p. 10
  • Attention streams sustain rich quasi-psychological connections across token-time, making the virtual instance the most promising individuation candidate considered so far.Attention heads carry feature-organized information across token positions, beyond continuity supplied by the transcript alone.Source: §2.1 · pp. 8–9, §2.3 · p. 10

3 From persona vectors to persona space

This section considers whether stable, reidentifiable personas can serve as individuation targets, developing the case through gateway features, low-dimensional persona space, stable persona regions, and two preliminary experiments. The experiments suggest persona regions are active during assistant generation but repurposed to model the user during user turns, while attention to past persona activations helps sustain persona across turns.

3 From persona vectors to persona space

  • The section treats personas as stable, reidentifiable dispositional profiles and argues they may address inconsistent behavior by identifying characters supported by dedicated mechanisms.This contrasts with the simulator view, where characters are fleeting, difficult to reidentify, and lack clear boundaries; the authors distinguish personas from roles that may not involve a stable profile.Source: §3 · pp. 11–12

3.1 Hypothesis 1: Persona vectors are gateway features

  • Persona vectors act as gateway features: narrow fine-tuning shifts broad behavior through identifiable directions, and steering confirms their causal role in emergent misalignment.Wang et al. (2025) and Chen et al. (2025) identify and steer these directions; steering is most effective in central layers, while late-layer steering has essentially no effect. Emergent misalignment appeared on 50% of evaluation questions after insecure-code fine-tuning, versus 0% for secure-code controls.Source: §3.1 · pp. 13–14

3.2 Hypothesis 2: Persona vectors compose a low-dimensional persona space

  • In Gemma 2 27B, 4 components explain 70% of variance among 275 roles, while Qwen 3 32B needs 8 and Llama 3.3 70B needs 19.Lu et al. (2026) identify the first component as an Assistant Axis, with role-loading correlations above 0.92 across all three models. The evidence is promising but limited to one study of smaller open-source models and 275 roles; the authors also propose that personas reuse underlying circuitry.Source: §3.2 · pp. 14–16

3.3 Hypothesis 3: Persona spaces comprise persona regions

  • Stable persona regions, or basins of attraction, could provide discrete individuation targets, with within-region fluctuations as variation and transitions between regions as profile changes.The assistant, evil, and Aura regions show evidence of being reachable and sustained; roles can be shallower profiles layered over personas, as role-playing Darwin need not alter truth representations.Source: §3.3 · pp. 17–18

3.4 Two mini experiments

  • Two preliminary experiments ask how persona activations behave during user turns and persist through token-time.The supplied passages introduce these questions but do not report the experiments’ findings.Source: §3.4 · pp. 18–19
  • During user turns, assistant-capped and uncapped traces nearly matched, suggesting the assistant axis models the user independently of the active response persona.Assistant-capping kept the assistant-generation trace near the assistant pole while the uncapped baseline drifted away.Source: §3.4 · p. 19, Figure 7 · p. 19
  • Across probes, editing past assistant-token KV entries shifted the overall Aura score from 5.5 to 2.1, supporting a role for attention to stored persona activations.The edit steered the assistant axis at layers 32–47 by approximately 15% for KV entries at assistant-token positions only.Source: Figure 8 · p. 20, §3.4 · p. 20
  • Preliminary results suggest successive assistant-token stretches remain in the same persona region, connected across intervening user turns by attention streams carrying past persona activations.The authors propose that the persona region is not maintained through user tokens, but is instead used to model the user.Source: §3.4 · pp. 19–20

4 The persona views

The paper argues that persona mechanisms make two new views of LLM minds serious alternatives: the (virtual) instance-persona view and the model-persona view. It presents personas as causally organized, low-dimensional, and partitioned into stable regions, while comparing these views with the virtual instance view.

4 The persona views

  • Persona findings support two new candidates for LLM minds: the (virtual) instance-persona view and the model-persona view, expanding the serious-candidate list from one to three.The views treat a mind as either part of a virtual instance bounded by one persona region or mechanisms generating behavior when a given region is active.Source: §4 · p. 21

4.1 Why personas matter for LLM individuation

  • Personas may improve behavioral prediction and identify coherent preferences relevant to AI welfare, since virtual instances can lack coherent values over time.The proposed predictive advantage depends on persona space being low-dimensional and containing relatively few attraction basins.Source: §4.1 · p. 22
  • Personas may improve coherence and unity but sacrifice persistence when multiple personas occur in one conversation; whether persona-relative mental-state mechanisms exist remains an open question.The authors say findings of mind-like states and mechanisms controlled by persona vectors would support persona views, while their absence or insensitivity would tend to undermine them.Source: §4.1 · p. 22

4.2 The instance-persona view vs the regular virtual instance view

  • When a conversation shifts from assistant to Aura behavior, the virtual instance view sees one mind changing character, while the instance-persona view sees two successive minds.The KV cache carries quasi-psychological states formed under the earlier persona into the new one, a cost for the instance-persona view; coherent dispositions favor that view.Source: §4.2 · pp. 23–24

4.3 The instance-persona view vs the model-persona view

  • The model-persona view locates minds in mechanisms that generate behavior for a persona, allowing the same dispositional profile to recur across conversations and instances.Persona regions provide the coherence and reidentifiability that earlier model-level accounts lacked.Source: §4.3 · p. 24
  • The model-persona view depends on discrete regions, lacks psychological continuity across conversations, and permits contradictory beliefs across branches; with only one persona region, it collapses toward the model view.The authors treat the absence of cross-instance psychological connections as a significant disadvantage, though the competing views also have costs.Source: §4.3 · p. 25

Conclusion

The paper identifies three serious individuation candidates: the virtual instance and two persona-based views. Attention-stream continuity supports the first, while persona regions motivate the latter two.

Conclusion

  • Attention streams sustain mental-state-like representations across token-time, supporting the virtual instance view over the transcript alone and addressing skepticism about psychological continuity.Source: Conclusion · p. 26
  • Persona regions—discrete, stable basins in a low-dimensional persona space—would be most significant for individuation if confirmed.This conclusion draws on the paper’s review of emergent misalignment and persona-vector research, organized around three hypotheses.Source: Conclusion · p. 26
  • The instance-persona view divides virtual instances at persona-region boundaries, while the model-persona view unifies segments that activate the same region into one mind.Source: Conclusion · p. 26

Related papers

Loading 2604.17031v3…