PaperlayerView exact revision on arXiv

Source-linked AI summary

Implicit Personality Representations in Humans and LLMs

Yilin Geng, Omri Abend, Eduard Hovy, Lea Frermann

arXiv:2609.12704v1cs.AI

TL;DR

The paper asks whether an LLM’s internal trait representations reproduce the relational structure of human personality impressions, rather than relying on the model’s questionnaire answers. It extracts trait directions from contrasting generations and compares their geometry with human impressions, finding aligned structure and dominant social and intellectual dimensions.

  • Problem

    Prior questionnaire-based personality profiles do not conclusively show how LLMs internally represent personality or enable comparison with human representational structure.

    Source: §1 · p. 2

  • Method

    The authors prompt contrasting behaviors for trait poles, extract activation differences as trait directions, and compare their structure with human personality impressions.

    Source: §1 · p. 2

  • Results

    The model recovers higher-order trait relations matching human impressions, with two dominant axes corresponding to social and intellectual dimensions.

    Source: §1 · p. 2, §6 · p. 9

  • Takeaways & Limitations

    The human-validated trait axes provide a shared space for projecting text about a person and measuring expressed personality against an external human reference.

    Source: §6 · p. 9

    Evidence comes from one model, and structural alignment does not establish that the model uses the matching geometry.

    Source: Limitations · p. 9

Abstract

from arXiv · show

A century of psychology has found that the trait words people use to describe one another vary, but the relational structure among those traits, which ones go together and which oppose, is strikingly consistent across raters and cultures. We test whether the LLM (Qwen 2.5-7B-Instruct) reproduces this structure in its internal trait representations. From millions of crowd-sourced personality ratings of fictional characters, we build a human implicit-personality matrix over hundreds of traits; from contrastive model activations, we build a matching matrix over the same traits. The two relational structures align strongly (Mantel r = 0.77), and the agreement holds trait by trait as well as in aggregate. Two dominant axes of the model's trait representations recover the social and intellectual dimensions long known to organize human personality impressions, social warmth and intellectual competence. On held-out dialogue, projecting model activations onto these directions yields personality profiles that agree with human ratings. This work establishes a framework that enables comprehensive, human-grounded comparison between internal model trait geometry and the shared structure of human personality impressions.

1 Introduction

The paper asks whether an LLM’s internal trait representations recover the relational structure found in human personality impressions. It compares contrastively extracted model directions with human ratings and reports strong alignment, including dominant social and intellectual dimensions.

1 Introduction

  • Prior work inferred model personality from questionnaire responses or output probabilities, but did not establish how internal trait representations compare with human structure.This distinction matters because model behavior may not accurately reflect internal mechanisms.Source: §1 · p. 2
  • Contrastive scenarios elicit opposite-pole behaviors whose activation differences form trait directions, which are compared with human representations for the same traits.The human reference comes from the crowd-sourced SWCPQ dataset.Source: §1 · p. 2
  • The model’s trait geometry aligns strongly with human personality-impression structure, and its two dominant axes correspond to social warmth and intellectual competence.The paper also compares projected profiles from held-out dialogue with the human reference.Source: §1 · p. 2
  • The framework provides a human-grounded coordinate system for measuring expressed personality against an external human reference.The authors suggest extending it to study personality development across narratives and related domains such as values and emotion.Source: §1 · p. 2

2 Related Work

Prior research studies personality through trait taxonomies, person-perception structure, model outputs, and internal concept directions. This paper compares the geometry of internal model trait representations with crowd-sourced human impressions.

2 Related Work

  • Human personality research includes both trait taxonomies such as the Big Five and HEXACO and person-perception dimensions such as social good–bad and intellectual good–bad.The latter are also described as warmth and competence.Source: §2 · p. 2
  • Prior work characterizes model personality through questionnaire answers or output probabilities, including recovered Big Five factors (Suh et al., 2024).Other studies use persona prompts to elicit trait-relevant behavior, while adjacent work examines opinions, values, and moral beliefs in model outputs.Source: §2 · p. 3

Concept directions and steering in the LLM

  • Concept directions have been identified and sometimes causally steered in model activations, while persona vectors contrast trait-expressing generations with neutral ones (Chen et al., 2025).Earlier work validates a single or small number of directions by their behavioral effects rather than characterizing their collective geometry.Source: Concept directions and steering in the… · p. 3
  • The paper compares internal LLM trait representations with crowd-sourced human impressions and reports aligned relational structures and matching warmth and competence axes.The comparison draws on representational similarity analysis and the Mantel test.Source: Concept directions and steering in the… · p. 3

3 Implicit Personality Representations from Human Data

The human reference is a map of shared personality impressions derived from crowd-sourced ratings of fictional characters. Trait profiles are compared across characters to construct a matrix of relationships among traits.

3 Implicit Personality Representations from Human Data

  • Volunteers use the SWCPQ to rate fictional characters on trait scales defined by two opposing poles.The released data contain 77.4 million ratings across 2,125 characters and 499 crowd-sourced trait scales.Source: §3 · pp. 3–4
  • Filtering yields 414 valid traits and ratings for 2,000 characters, retaining over 99% of raters and ratings while removing low-coverage pairs and unsuitable traits.Traits must have enough characters rated near both ends of the 1–100 scale and describe dispositional or behavioral constructs.Source: §3 · p. 4
  • Each trait is represented by its mean rating profile across the same 2,000 characters, allowing profiles for different traits to be compared directly.Each character–trait mean aggregates a median of 26 raters, with an interquartile range of 14–62.Source: §3 · p. 4
  • The human implicit-personality matrix records Pearson correlations between trait profiles, so related traits tend to receive similar rating patterns across characters.The reference measures shared impressions, not fictional characters’ objective or “true” dispositions.Source: §3 · p. 4
  • Across 100 random respondent-session splits, separately constructed trait matrices agree at mean Mantel r = 0.998, while human–model alignment changes little when the minimum cell count rises from 5 to 20: 0.765 versus 0.758.Individual trait-rating profiles have mean split-half r = 0.865; the authors describe the map as a data-driven reference built without a prior theoretical taxonomy.Source: §3 · p. 4

4 Implicit Personality Representations from LLM Activations

The paper derives model trait directions from responses to contrasting trait poles and constructs a relational matrix from those directions. It validates generated responses and tests the stability of the resulting geometry under alternative selection procedures.

4 Implicit Personality Representations from LLM Activations

  • For each trait, contrasting scenarios elicit pole-specific responses whose internal activations are recorded and differenced to produce a trait representation.The trait pole is paired with a representative SWCPQ character because using only the trait term suppressed answers for negative traits.Source: §4 · p. 4
  • On 220 matched traits, trait-only extraction reaches Mantel r = 0.635, compared with r = 0.775 for character+trait extraction; no one of 10,000 permutations reaches either statistic (p < 10−4).Character names improve extraction yield but are not necessary for substantial human–model alignment.Source: §4 · p. 5
  • The model generates three completions for each of ten scenarios and five characters per pole, and a judge retains responses with scores above 70 or below 30.Scenario rubrics check pole discrimination, avoid explicit mention of the poles, and screen for inherent scenario bias.Source: §4 · p. 5
  • The recovered geometry remains stable under further score-threshold and fixed-count re-selection within responses passing the primary 30/70 filter.This robustness analysis is conditional on the initial capture filter.Source: §4 · p. 5
  • Human annotators agree with one another on 76% of responses, while the judge agrees with them on 73%, at human inter-rater level.The study collects judgments on 100 responses, with four annotators per text.Source: §4 · p. 5

From ratings to the LLM personality matrix

  • For each pole, response-token activations are averaged at layer 204, and the difference between pole means forms the trait vector used in the matrix and later analyses.The resulting LLM matrix compares trait directions across residual-stream coordinates, abstracting scenario noise.Source: From ratings to the LLM personality… · p. 6

5 Results

The human and model trait matrices show strong overall and trait-by-trait alignment, and the model’s dominant dimensions match established human personality-impression dimensions. On held-out character dialogue, projected profiles also agree with human ratings, though speech persona can diverge from rated disposition.

5.1 Alignment of M(H) and M(L)

  • Per-trait alignment has median 0.80 and mean 0.74; weakest matches cluster around midpoint ratings or context-specific and vague traits.The model reproduces human structure trait by trait, with disagreement concentrated among traits carrying little reliable human signal.Source: Figure 3 · p. 6, §5.1 · p. 7
  • rMantel = 0.765 (p<10−4), and remains 0.708 after near-synonymous trait comparisons are removed.The cluster-controlled result is based on 58,623 dyads (79.3%).Source: §5.1 · p. 6

5.2 Principal Axes of Personality Representation

  • The first two principal components explain 59% of variance, organizing traits around intellectual competence and interpersonal warmth, as in human impression research.The component structure closely matches Rosenberg et al. (1968), despite construction without predefined taxonomies or inter-trait similarity information.Source: §5.2 · pp. 7–8, Figure 4 · p. 7

5.3 Personality Inference

  • Across 256 held-out characters, median profile agreement is r = 0.403 on 195 aligned pairs, and 93.0% of characters have positive agreement.Across all 385 traits, median agreement is r = 0.328 (95% CI [0.311, 0.359]); the result supports transfer beyond construction scenarios.Source: §5.3 · p. 8, Figure 5 · p. 8
  • Negative profile agreement occurs when a character’s speech persona diverges from the disposition reflected in human ratings.The framework also proposes tracking how a profile converges as more of a character is seen, and how disposition changes across a narrative.Source: §5.3 · p. 8

6 Conclusion

Qwen 2.5-7B-Instruct recovers the higher-order organization of human trait impressions, including the social and intellectual dimensions. The authors present the resulting trait space as a human-grounded coordinate system for measuring expressed personality.

6 Conclusion

  • The model recovers higher-order trait relations and the two dominant axes associated with social and intellectual dimensions in human personality impressions.Trait directions were extracted one at a time from contrastive behavior, without inter-trait information supplied; the structure matches one built independently from millions of human ratings.Source: §6 · p. 9
  • The authors interpret the alignment as evidence that trait structure may be latent in language, while noting that convergence across other language-trained systems remains a hypothesis.They suggest other systems trained to compress human language may converge on the same low-dimensional axes.Source: §6 · p. 9
  • The human-validated trait axes let text about a person be projected into a shared space and compared with an external human reference.The authors say the framework has begun to be applied to persona, revelation, and change in text.Source: §6 · p. 9

Limitations

The evidence is limited to one model, and structural agreement does not establish that the model uses the matching geometry. The judge-based exemplar filter is also imperfect.

Limitations

  • Evidence comes from one model, leaving the hypothesized convergence across systems untested.The comparison is structural rather than causal and does not test whether the model uses the matching geometry; the judge filter also matches human labels imperfectly (precision 0.79, recall 0.82).Source: Limitations · p. 9

A Human Data details

The human trait matrix is built from cleaned crowd-sourced ratings, with checks on the reproducibility of aggregate impressions and sensitivity to cell counts.

A Human Data details

  • Exact-duplicate pole text on two source scales led the authors to merge their ratings because raters were answering the same question.Source: A Human Data details · p. 11
  • Rating-level and rater-level cleaning removes responses with extreme response times and raters with too few surviving ratings, very fast median times, or low rating variability.After cleaning, 99.9% of source raters and 99.7% of source ratings remain.Source: A Human Data details · p. 11
  • The trait-inclusion criteria exclude scales probing demographics, gross physical attributes, occupational or socioeconomic categories, category archetypes, and identity labels.The excluded-scale list and category annotations are released with the data; an LLM judge identifies these traits.Source: A Human Data details · p. 11
  • The human trait matrix is highly reproducible across 100 independent splits of respondent sessions, with mean Mantel r = 0.998388.Individual trait-rating profiles are less consistent but remain reliable (mean split-half r = 0.865); the authors caution that these checks establish shared impressions, not accuracy against objective character personality.Source: A Human Data details · pp. 11–12
  • Raising the minimum ratings per character–trait cell from five to 20 leaves human–model alignment nearly unchanged, from 0.764651 to 0.758018.At those thresholds, 91.6% and 68.9% of cells remain, respectively; synonym-cluster-controlled alignment also changes only modestly, from 0.707741 to 0.700290.Source: A Human Data details · pp. 11–12

B.1 Suppression Under the Trait-Only Prompt Method

Prompt framing affects which trait directions can be extracted and how strongly their geometry aligns with human impressions. The appendix tests prompt controls, judge filtering, exemplar re-selection, and layer sensitivity.

B.1 Suppression Under the Trait-Only Prompt Method

  • Across 402 scenario-valid scales, character-and-trait prompts retain 96%, character-only 77%, and trait-only 63%; character role-play also reduces harmful-pole suppression.The caption describes the yield table; the prose reports that character role-play cuts harmful-pole collapse to 5 of 54 loaded scales, and adding the trait removes it entirely.Source: Table 1 · p. 12, §B.1 · p. 12

B.2 Prompt-Method and Character-Anchor Controls

  • Table 2 reports human–model alignment for the matched prompt-method cohort.Source: Table 2 · p. 12
  • Different character-pair directions for the same trait have median similarity r = 0.595, versus r = 0.003 for random cross-trait pairs.For traits with five directions, a held-out pair compared with the other four has median r = 0.747; the authors conclude the extracted signal is substantially trait-specific.Source: §B.2 · p. 12

B.3 Scenario Generation Prompts

  • Scenario prompts are designed so opposite trait poles lead to distinct, equally available responses without signaling the target trait or making one response easier.Source: §B.3 · p. 13

B.4 LLM Judge Details

  • The judge filter keeps 79% of retained exemplars as matching the prompted pole by human consensus and recovers 82% of human-confirmed expressions.The authors use the judge as a coarse three-bin filter and note that it trades recall for precision; it is not perfect.Source: §B.4 · pp. 13–14

C Layer Sweep details

B.5 Sensitivity to Exemplar Re-Selection

  • Further exemplar re-selection tests robustness only within responses already admitted by the primary 30/70 filter, so it does not rule out effects of that initial filter.Judge-score separation is associated with per-trait human–model alignment (Pearson r = 0.312, p = 3.9 × 10−10, n = 385).Source: §B.5 · p. 14

D Similarity Metric Robustness of the Alignment Statistic

The model–human alignment statistic is robust to using cosine similarity instead of Pearson correlation. The model-side matrices are effectively identical under the two choices.

D Similarity Metric Robustness of the Alignment Statistic

E Activation-Space Diagnostics

Activation-space diagnostics show that the human–model alignment is not solely dependent on a common mean direction or PC1, although leading components carry substantial structure. PCA also identifies social and intellectual axes among the dominant components, with further examples of dialogue projections and failures.

E Activation-Space Diagnostics

  • Subtracting the mean direction changes Mantel alignment only from r = 0.764731 to r = 0.764494.The mean norm is 6.8% of the mean individual-vector norm, and mean pairwise cosine similarity is 0.003; this rules out domination by a common mean, not isotropic covariance.Source: E Activation-Space Diagnostics · p. 15
  • Removing the PC1 projection leaves Mantel r = 0.613537, and no one of 10,000 permutations reaches either processed-matrix statistic (p < 10−4).PC1 explains 36.6% of trait-direction variance; the alignment is not solely dependent on the global mean or PC1, although leading components carry substantial structure.Source: E Activation-Space Diagnostics · p. 15

H Additional projection-probe examples

  • Figure 9 shows held-out dialogue-projection examples spanning heroes and villains, with r = 0.50 to 0.73 on 195 aligned trait pairs.Figure 10 shows three failures with r < 0; blue marks the pole humans rate and red the pole the dialogue projects.Source: H Additional projection-probe examples · p. 15

G Principal Component Analysis Details

  • PC1 and PC2 capture 37% and 22% of variance, respectively, or 59% jointly; 90% is reached only by PC21.The two leading components correspond to the intellectual and social axes, and stand clear of the rapidly decaying tail.Source: Figure 7 · p. 15

H.1 Full-Trait Evaluation and Null Baseline

The held-out dialogue evaluation compares projected profiles with complete human-rated profiles across 256 characters, using a character-resampling interval and a shared trait-axis permutation null. Most character-level correlations are positive, and the evaluation tests representational transfer rather than causal use during generation.

H.1 Full-Trait Evaluation and Null Baseline

  • The held-out cohort has no identifier overlap with the 877 characters used as anchors for the final directions, and no one of 10,000 null medians reaches the observed value.The null applies one shared random permutation of human trait axes across characters; character rankings are stable between full and aligned evaluations (Spearman ρ = 0.966, n = 256). The analysis tests transfer, not causal use during generation.Source: §H.1 · p. 15
  • Top-aligned trait scales have clear behavioral signatures, while bottom-aligned scales reflect diffuse stereotypes or poorly differentiated dimensions.The table describes the alignment gradient as reflecting how sharply each trait dimension is shared across raters.Source: Table 5 · p. 16
  • 93% of the 256 held-out characters have positive dialogue-projected versus human-rated profile correlation, with median r = 0.40.Source: Figure 8 · p. 16
  • Figure 9 plots standardized scores for 195 aligned trait axes against the y = x line, with each panel reporting Pearson r for one held-out character.Source: Figure 9 · p. 17

Related papers

Loading 2609.12704v1…