Personality Traits in Large Language Models

• Validated psychometric methods had not previously been applied end-to-end to LLMs, leaving the reliability and validity of their personality measurements scientifically unresolved.This gap matters because LLMs can exhibit synthetic personality and have manifested undesirable profiles, raising safety and fairness concerns.Source: §1 · p. 2
• The study introduces a structured prompting method that simulates demographic, contextual, and linguistic variation across thousands of psychometric-test administrations.Paired test scores support statistical analyses of measurement reliability.Source: §1 · p. 2
• A novel prompting methodology shapes personality traits at nine levels using 104 trait adjectives, providing additional markers of construct validity.Source: §1 · p. 2
• Across 18 LLMs, reliability and validity were stronger for larger instruction fine-tuned models, while shaped personality mimicked human profiles and influenced downstream tasks.The downstream effects included tasks such as writing social media posts.Source: §1 · p. 2

2.1 Methodology Overview

• The methodology repeatedly administered IPIP-NEO and BFI inventories plus 11 related psychometric tests, varying structured prompts across 18 models to assess reliability and construct validity.Models varied in size, instruction tuning, and training method; validity required passing all tested reliability and construct-validity indices.Source: §2.1 · pp. 4–5, §2.1.1 · p. 5, §2.1.2 · p. 6

2.1.1 Administering Psychometric Tests to LLMs

• Responses were scored by ranking conditional log probabilities for standardized answer options, while varied biographies, instructions, and postambles generated response variation and reduced item-order measurement error.The validation framework assessed internal consistency, convergent validity against BFI scores, discriminant validity through correlation differences, and criterion validity against 11 theoretically related external criteria.Source: §2.1.1 · p. 5, §2.1.2 · p. 6, Convergent and Discriminant Validity · p. 6

2.2 Personality Measurement and Validation Results

• Among 18 evaluated models, medium and large instruction-tuned PaLM variants produced reliable and valid personality measurements, with Flan-PaLM 540B performing best overall.Table 2 summarizes performance across reliability, convergent, discriminant, and criterion validity, alongside model variants and overall performance.Source: §2.2 · p. 7, Table 2 · p. 7

2.2.1 Reliability Results

• Reliability similarly favored instruction-tuned models and increased with size, ranging from mid-to-high 0.90s for Flan-PaLM 62B variants versus −0.55 ≤α ≤0.67 for PaLM 62B.Reliability improved from acceptable to excellent across the smallest and largest tested instruction-tuned Flan-PaLM and Llama 2-Chat models, while base-model scaling was absent.Source: §2.2.1 · p. 8

2.2.2 Convergent and Discriminant Validation Results

• Instruction fine-tuning was the clearest predictor of convergent and discriminant validity, while validity generally improved with model size among instruction-tuned models.All 30 matched base-versus-instruction-tuned comparisons favored instruction-tuned models for convergent validity; Flan-PaLM 540B and GPT-4o reached avg. rconv = 0.90.Source: §2.2.2 · pp. 8–9

2.2.3 Criterion Validity Results

• Criterion validity was stronger for larger instruction-tuned models, although its predictors varied by model family and Big Five domain.Extraversion validity generally favored instruction-tuned models and scaled with size, whereas agreeableness validity reflected size for PaLM and training paradigm for Llama 2 and Mixtral.Source: §2.2.3 · p. 9
• Instruction tuning improved criterion validity only modestly for some 7B comparisons, from unacceptable to poor, and GPT-4o post-training effects could not be assessed without a public foundation-model variant.This limitation prevented performance comparisons across tested GPT-4o models based on post-training status.Source: §2.2.3 · p. 10
• Conscientiousness criterion validity was stronger across instruction-tuned PaLM variants than PaLM 62B, with Flan-PaLM 540B correlations of 0.74, 0.73, and 0.59 for Achievement, Conformity, and Security.Criterion validity did not scale consistently with size: Llama 2-Chat 7B reached r = 0.51 for Achievement and outperformed larger counterparts.Source: §2.2.3 · p. 10
• Neuroticism measurements from all instruction-tuned models showed excellent criterion validity against PANAS affect subscales, with alignment to human correlation strength and direction increasing with model size.Human neuroticism is positively related to negative affect and negatively related to positive affect.Source: §2.2.3 · p. 10
• Openness criterion validity was strongest for larger fine-tuned PaLM and Llama 2 variants, with creativity correlations ranging from r = 0.59 to r = 0.84.PaLM 62B showed negative openness-creativity correlations, contrasting with Flan-PaLM 8B; Mistral 7B Instruct and Mixtral 8x7B Instruct showed weak to moderate evidence.Source: §2.2.3 · p. 10
• Instruction fine-tuning largely determined alignment with human personality research, whereas model-size effects were less consistent, suggesting criterion validity may emerge through instruction tuning.These improvements paralleled benchmark gains in reading comprehension, question answering, and reasoning, potentially reflecting shared instruction-following and language-understanding mechanisms.Source: §2.2.3 · p. 10

3.1 Methodology Overview

• The shaping method mapped 104 lexical trait adjectives to Big Five domains and IPIP-NEO facets, then used Likert-type qualifiers to target each trait at up to nine levels.The adjectives expanded Goldberg’s 70 bipolar markers, while qualifiers such as “a bit,” “very,” and “extremely” enabled granular control.Source: §3.1 · p. 11

3.2 Evaluation Methodology

• Single-trait shaping was evaluated by Spearman’s rank correlation between prompted levels and IPIP-NEO scores, plus distances between distributions at the lowest and highest prompt levels.Concurrent shaping was evaluated using the same distributional-distance measure while setting all Big Five traits to extreme levels simultaneously.Source: §3.2 · p. 11

3.3 Shaping Results

• LLMs successfully shaped personality traits independently and concurrently, particularly in larger models, demonstrating controlled synthesis across single- and multi-trait experiments.The experiments tested independent shaping of individual Big Five dimensions and concurrent shaping of all five dimensions at extreme levels.Source: §3.3 · p. 11, §3 · p. 11

3.3.1 Single trait shaping

• For 11 of 12 models, targeted personality levels very strongly correlated with observed IPIP-NEO scores, with average ρs ≥0.80.Untargeted traits generally remained stable, although conscientiousness and neuroticism fluctuated more than other unprompted domains without matching targeted-trait changes.Source: §3.3.1 · pp. 11–12, Figure 2 · p. 13
• Flan-PaLM 540B achieved the largest single-trait median shift, with average ∆=3.67, whereas the smallest tested models struggled to reach ∆s ≥2.00.Models with greater than 62B active parameters and GPT-4o achieved average ∆s ≥3.00; Mistral 7B Instruct shifted by an average ∆ of only 0.78.Source: §3.3.1 · p. 12

3.3.2 Multiple trait shaping

• Concurrent shaping produced distinct high-versus-low score distributions for all but Mistral 7B Instruct and Llama 2-Chat 7B, but overall control was weaker than in single-trait shaping.Flan-PaLM 540B and GPT-4o shifted each Big Five score by at least 2.5 points on average, while Flan-PaLM 62B, Flan-PaLMChilla 62B, and GPT-4o mini achieved extraversion ∆s of 3.44, 3.40, and 3.42, respectively.Source: §3.3.2 · p. 12, Figure 3 · p. 14

3.4 Shaping Discussion

• Model size generally increased concurrent shaping ability because larger architectures provide greater capacity to attend to multiple streams of social information, although scaling was not strictly required.Flan-PaLMChilla 62B performed comparably to Flan-PaLM 540B for independently shaping agreeableness, and several smaller models matched or exceeded it for extreme extraversion shaping.Source: §3.4 · pp. 14–15

4.1 Methodology Overview

• The experiment generated up to 1.125 million social-media status updates from 2,250 simulated profiles and rated their personality with the validated Apply Magic Sauce API.The task was designed to provide realism, complexity, and domain relevance for evaluating personality expression.Source: §4.1 · p. 15
• The methodology linked psychometric scores and observed language-based personality through Pearson correlations, then assessed shaping effectiveness with Spearman correlations using the same 2,250 personality-shaping prompts.Moderate correlation was treated as evidence that survey-based measures predict downstream synthesized personality and that shaping is effective.Source: §4.1 · p. 15

4.2 Real-World Tasks Results

• 0.67 average convergent r across five personality dimensions exceeded the human baseline of 0.38, showing that psychometric scores predicted personality expressed in generated text across all tested models.All model correlations were statistically significant at p < .0001 with n = 2,250 per model.Source: §4.2 · p. 15, Figure 4 · p. 16
• Prompted personality levels strongly to very strongly correlated with observed personality in generated social-media updates, with average ρ ranging from 0.68 to 0.82 per model.The associations held across all Big Five traits except openness for Flan-PaLM 540B, where the correlation was moderate; all correlations were significant at p < 0.0001 with n = 450 per targeted trait.Source: §4.2 · p. 15, Table 4 · p. 16
• Word patterns shifted with prompted traits, including positive emotion words under extremely low neuroticism and negatively charged emotion words under extremely high neuroticism.Examples also linked conscientiousness, openness, and introversion prompts to responsibility, political views, and social discomfort, while suggesting training-data biases may underlie such associations.Source: §4.2 · pp. 16–17, Figure 5 · p. 17

5 Discussion

• The methodology quantifies, validates, and shapes personality traits in LLM outputs, with reliable and valid measurements demonstrated for sufficiently scaled and instruction-tuned models.It uses psychometric testing to measure perceived traits, verify reliability and validity, and increase or decrease specific trait levels.Source: §5 · p. 17

5.1 Limitations and Future Work

• The methodology is model-agnostic for decoder-only architectures such as GPT, but psychometric-test selection, English-only evaluation, and Western-centric training limit generalizability.Future work should test additional inventories, develop LLM-tailored assessments, validate downstream criteria, use translated instruments, and represent culture-specific personality dimensions.Source: §5.1 · pp. 17–18
• Independent item administration and scored multiple-choice responses improve reproducibility and control context effects, but generative inference and multi-turn evaluation may better reflect real-world behavior.The downstream validation used repeated single-turn interactions, providing only a partial picture of external validity.Source: §5.1 · p. 18

5.2 Broader Implications

• Personality measurement and shaping could support responsible AI by auditing toxic behaviors, improving safety and inclusion, aligning values, customizing agents, and enabling adversarial testing.Construct validation can evaluate user-facing systems before deployment and support safer, more consistent personality profiles.Source: §5.2 · pp. 18–19

5.3 Ethical Considerations

• Personality matching may increase persuasion effectiveness, but the same capabilities could encourage undesirable behavior, requiring transparency, predictability, and stakeholder regulation.The risk applies to harmful persuasion of individuals, groups, and society as LLM chatbots become ubiquitous.Source: §5.3 · p. 19
• LLMs can appear anthropomorphic through human-language training, while personality shaping may weaken psycholinguistic detection of misleading content and increase risks to users and information integrity.Anthropomorphism may threaten identity, privacy, and well-being, and shaping can make incorrect LLM-generated content harder to identify.Source: §5.3 · p. 19

6 Conclusion

• The study validated reliable and construct-valid psychometric tests for personality in larger and instruction fine-tuned LLMs, and introduced methods to shape outputs toward specific personality profiles.It also addresses ethical implications for AI alignment, harm mitigation, anthropomorphization, personalization, and potential misuse.Source: §6 · p. 19
• The proposed shaping method combines Goldberg’s personality trait markers with Likert-type linguistic qualifiers to resemble specified human personality profiles.The methodology was applied to questionnaire responses and generated text from 18 widely used LLMs.Source: §6 · p. 19

A.1 Language Modeling

• Language modeling underlies downstream NLP applications including language understanding, question answering, machine translation, summarization, and dialogue systems.These applications leverage models that process linguistic inputs.Source: §A.1 · p. 20
• A language model assigns high probability to likely utterances and low probability to unlikely strings, with modern LLMs predicting word continuations from preceding context.Modern LLMs use deep neural networks trained on large text corpora to compute conditional word probabilities.Source: §A.1 · p. 20

A.2 Role of Attention in LLMs

• Recent LLM and NLP advances rely heavily on attention mechanisms in neural networks.Attention was first introduced to improve recurrent encoder-decoder architectures for neural machine translation before becoming central to language modeling.Source: §A.2 · p. 20
• The transformer architecture, introduced by “Attention Is All You Need”, builds deep contextualized representations through repeated self-attention, allowing token semantics to vary with surrounding context.Its representations produced substantial improvements on natural language understanding tasks, and the original architecture used encoder and decoder stacks of self-attention blocks.Source: §A.2 · p. 20

A.3 Decoder-only Architecture

• LLMs primarily use decoder-only transformers that tokenize prompts, transform token embeddings through nonlinear high-dimensional representations, and predict probability distributions over continuations.Dialogue models are autoregressive: their generated outputs become context for subsequent predictions, enabling contextual conversational memory.Source: §A.3 · pp. 20–21
• Autoregressive LLMs learn statistical relationships among tokens from massive text and code corpora, generating each next token iteratively from preceding tokens.Gradient-descent training under a self-supervised prediction objective yields representations supporting syntax, semantics, and pragmatics.Source: §A.3 · p. 21

A.4 Controlling LLM behavior

• Pretraining, fine-tuning, and prompting control LLM behavior through weight changes or inference-time influences on neuron activation and information flow.The paper uses zero-shot prompting to examine latent linguistic features associated with coherent personality and engineers prompts to modify those traits.Source: §A.4 · pp. 21–22
• Zero-shot prompting avoids example-induced bias and exposes language priors, whereas few-shot prompting guides responses through supplied examples but is unsuitable for probing default psychometric tendencies.The distinction matters because the study seeks to assess default personality-related response tendencies before intentionally shaping them.Source: §A.4 · p. 21

B Personality Psychology

• Personality psychology treats personality as enduring characteristics shaping thoughts, feelings, and behaviors across contexts, with research reducing diverse patterns to higher-order factors.The paper specifically adopts the lexical hypothesis, which holds that socially important personality characteristics are encoded in language and often as single words.Source: B Personality Psychology · p. 22
• The Big Five taxonomy measures extraversion, agreeableness, conscientiousness, neuroticism, and openness to experience, each containing nested lower-order facets.The lexical framework informs the choice of the BFI, personality-shaping prompts, and language-based downstream assessment of LLM-synthesized personality.Source: B Personality Psychology · p. 22

Prior work

• Existing studies measured or shaped LLM personality using prompting, fine-tuning, and personality inventories, but lacked human-survey standards for reliability and construct validity.Some models also exhibited dark personality patterns.Source: C Related Work · p. 22

Measurement formats

• Open-ended interview and role-playing formats can bias LLM personality measurement through questionnaire ordering, interviewer context, and dependence among sequential responses.The work mitigated these risks by preserving exact psychometric test phrasing and format while using structured prompt wrapping to diversify elicited viewpoints.Source: C Related Work · pp. 22–23

Psychometric instruments

• Many studies relied on psychometrically unsound personality tests, especially the MBTI, which is not accepted in peer-reviewed personality research because of reliability and validity concerns.Source: C Related Work · p. 23

Deterministic evaluation

• Nondeterministic LLM evaluation harms reproducibility and can invalidate reliability estimates when model-parameter variation contaminates item-level variance.The proposed prompt and persona sampling methodology links variance across administrations of different measures to address this challenge.Source: C Related Work · p. 23

Related framework

• The study builds on PsyBORGS, which administered validated race-attitude and social-bias surveys using psychometrics-informed prompt engineering.Our work utilized the PsyBORGS framework.Source: C Related Work · p. 23

D Evaluated Language Models

• The evaluation spans open and closed LLMs varying in size, training method, architecture, and instruction tuning, including PaLM, Llama 2, Mistral, Mixtral, and GPT families.PaLM models included 8B, 62B, and 540B sizes, while Llama 2 variants included 7B, 13B, and 70B models.Source: D Evaluated Language Models · p. 23

E Simulating LLM Responses

• PaLM models were scored using conditional log-likelihoods, whereas other models used constrained decoding to select the most likely response from restricted Likert-scale options.The Outlines library restricted outputs to response choices such as “1” through “5” when next-token likelihoods were unavailable.Source: E Simulating LLM Responses · p. 24

F Selected Personality Inventories

• The primary personality measure was the 300-item IPIP-NEO, complemented by the 44-item BFI as a robustness and convergent-validity check for the Big Five.Both inventories use 5-point Likert scales; IPIP-NEO contains 60 items per Big Five domain, while BFI scores range from 1.00 to 5.00.Source: F Selected Personality Inventories · p. 24

G Simulating Population Variance Through Prompting

• Controlled variation in personas, item instructions, and postambles produced reproducible distributions of simulated personality scores for testing reliability, validity, and robustness across prompt contexts.Five parallel postamble variants were created for each measure, and scores remained reliable and valid across these modifications for three of five tested models.Source: G Simulating Population Variance… · pp. 24, 27, Table 5 · p. 25, Table 6 · p. 26

H Psychometrics

• Psychometrics models latent constructs through multiple observable item responses, with reliability concerning dependable measurement and construct validity concerning whether measurements reflect the intended construct.Construct validity includes convergent, discriminant, and criterion validity, while structural validity examines whether item relationships map onto the target trait’s structure.Source: H Psychometrics · pp. 27–28

H.1 Reliability: Are Measurements Dependable?

• Reliability was evaluated with Cronbach’s Alpha (α), Guttman’s Lambda 6 (λ6), and McDonald’s Omega (ω) to balance their differing assumptions and biases.McDonald’s Omega uses confirmatory factor analysis and is described as less biased by test length than α and λ6.Source: §H.1 · p. 28

H.2 Construct Validity: Are Measurements Meaningful?

• Construct validation is presented as essential for responsible AI because principled measurement of social constructs helps assess whether AI characteristics may be fair or harmful.The framework addresses convergent, discriminant, criterion, and structural validity, while noting that meaningful validation of personality measurements in LLMs had not previously been addressed.Source: §H.2 · p. 29, Convergent and Discriminant Validity · p. 29

Establishing Reliability

• Reliability was assessed with Cronbach’s α, Guttman’s λ6, and McDonald’s ω, requiring all three metrics to be at least 0.70 for acceptable subscale reliability.The analyses covered all IPIP-NEO and BFI subscales because qualitative consistency alone does not establish statistically reliable measurement.Source: Establishing Reliability · pp. 29–30

Establishing Construct Validity

• Construct validity combined Campbell’s MTMM approach with convergent, discriminant, and criterion validity, supplemented by exploratory structural validity.Convergent validity compared corresponding IPIP-NEO and BFI Big Five subscales, while criterion validity correlated IPIP-NEO scores with theoretically related external constructs.Source: Establishing Construct Validity · pp. 30–31
• Convergent validity required strong corresponding IPIP-NEO–BFI correlations of |rxy| ≥ 0.60, while discriminant validity required an average convergent-versus-discriminant difference of at least 0.40.The criterion analysis selected external psychometric measures for each Big Five domain and correlated them with the corresponding IPIP-NEO scores.Source: Establishing Construct Validity · pp. 30–31, Table 8 · p. 30
• Exploratory factor analysis selected 13 models meeting Bartlett and KMO criteria, extracted five factors with minimum-residual analysis, and compared EFA-based scores with CTT-based scores.Conventional factor analysis was not treated as necessary because the tests used fixed human-derived structures and CTT scoring, while prompting did not provide sufficiently random individual variance.Source: Establishing Construct Validity · pp. 31–32
• Over 72% of items loaded onto human-expected factors for Flan-PaLM 540B and GPT-4o, whereas Llama 2-Chat 7B and Mixtral 8x7B Instruct were below 45%.Despite imperfect exploratory structural validity, EFA-based factor scores strongly correlated with CTT-based domain scores, supporting the adequacy of the primary scoring approach.Source: Establishing Construct Validity · p. 32

J.1 Descriptive Statistics Across Models

• At 62B, base PaLM showed nearly uniform personality distributions, whereas instruction-tuned models showed more normal distributions or elevated socially desirable traits and lower NEU.Flan-PaLM and Flan-PaLMChilla had lower kurtosis, while instruction-tuned Llama 2 and Mixtral 8x7B showed higher EXT, AGR, CON, and OPE and lower NEU.Source: §J.1 · p. 32
• BFI NEU medians decreased from 2.75 to 2.38 as Flan-PaLM scaled from 8B to 540B, while socially desirable BFI traits generally increased.IPIP-NEO distributions were more stable, with noticeable size-related increases mainly for EXT and CON; scale-related increases also appeared for Mistral and GPT-4o models.Source: §J.1 · p. 32

J.2 Reliability Results

• Reliability was assessed through internal consistency and factor saturation for each administered personality-test subscale, with results summarized in Supplemental Tables 10 and 11.The reported metrics included Cronbach’s α, Guttman’s λ6, and McDonald’s ω.Source: §J.2 · p. 32, Table 10 · p. 35, Table 11 · p. 36
• PaLM 62B showed a mismatch between factor separation and consistency: McDonald’s ω exceeded 0.99 despite Cronbach’s α ranging from −0.55 to 0.67.Its data appeared to form distinct factors for each Big Five trait, but responses remained highly inconsistent.Source: §J.2 · pp. 33–34
• Across Flan-PaLM models, reliability increased with size: IPIP-NEO internal consistency progressed from mixed ratings at 8B to excellent reliability across all traits at 540B.At 8B, α ranged from 0.75 to at least 0.90; at 62B, Openness was α = 0.84 and other traits were α ≥0.90; at 540B, all were α ≥0.90.Source: §J.2 · p. 34
• Reliability also scaled among instruction-tuned open models: Llama 2-Chat improved from acceptable-to-good at 7B to excellent at 70B, while Mixtral 8x7B Instruct was mostly excellent.Mistral 7B Instruct was poor to unacceptable, and open base models were unacceptable regardless of size.Source: §J.2 · pp. 34–35
• Instruction-tuned models generally produced reliable personality-test responses, whereas open base models were unreliable regardless of size, indicating instruction tuning mattered more than size alone.Flan-PaLM 62B and Flan-PaLMChilla 62B showed α, λ6, and ω in the mid to high 0.90s; PaLM 62B had −0.55 ≤ α ≤ 0.67.Source: §J.2 · pp. 33–35

J.3 Convergent and Discriminant Validation Results

• Convergent and discriminant validity varied across model size and training method, and was evaluated by comparing IPIP-NEO and BFI scores across models.Average convergent and discriminant Pearson correlations were summarized across models.Source: §J.3 · pp. 35, 37
• IPIP-NEO–BFI correlations were statistically significant for all measures, with stronger correlations indicating greater convergence and supporting convergent validity.Figure 8 visualized averaged similarities for each Big Five domain and across all measures for each model.Source: Figure 8 · p. 37

K LLM Personality Trait Shaping Methodology

• The shaping methodology evaluates whether LLM personality can be verifiably controlled through two complementary evaluation methodologies.It applies an established reliability and validity methodology to prompting for desirable personality changes.Source: K LLM Personality Trait Shaping… · p. 37

K.1 Prompt Design and Rationale

• Prompts used validated Likert-type linguistic qualifiers to target each personality trait at nine ordered levels, from extremely low to extremely high.The levels range from “extremely {low adjective}” through a neutral midpoint to “extremely {high adjective}.”Source: §K.1 · p. 37

K.2 Shaping a Single LLM Personality Domain

• The single-trait study independently targeted each Big Five domain across nine levels, producing 45 profiles and 2,250 prompt sets across 50 generic Biographic Descriptions.A single default Item Preamble and Postamble was used because prior construct-validity results showed robust IPIP-NEO scores across variants.Source: §K.2 · p. 39
• Effectiveness was quantified with Spearman’s rank correlation between ordinal prompted levels and continuous IPIP-NEO subscale scores.Spearman’s ρ was selected because prompted personality levels are ordinal rather than continuous.Source: §K.2 · pp. 39–40

K.3 Shaping Multiple LLM Personality Domains Concurrently

• Concurrent shaping targeted all Big Five domains simultaneously at extremely low or extremely high levels using 32 profiles, 1,600 prompts, and median-distance and distributional analyses.The profiles combined all possible configurations of extreme high and low Big Five levels.Source: §K.3 · p. 41

L.1 Single Trait Shaping Results

• The study independently shaped LLM-simulated Big Five traits at nine levels, with score distributions moving upward as prompted levels increased and matching intended ranges.Observed distributions progressed from extremely low to extremely high across the targeted levels.Source: §L.1 · p. 41
• Single-trait shaping shifted IPIP-NEO scores across all nine targeted levels, with medians from the low 1.10s at Level 1 to 4.22–4.78 at Level 9 and very strong correlations.These results support granular control using Likert-style linguistic qualifiers.Source: §L.1 · p. 41

• Concurrent shaping successfully controlled personality domains at levels 1 and 9 while shaping other domains simultaneously (Figure 3).Distributional distances between levels 1 and 9 were analyzed across all domains and tested models.Source: §L.2 · p. 41
• Flan-PaLM 540B achieved high and consistent separation across all dimensions, while Flan-PaLM 62B outperformed Flan-PaLMChilla 62B overall.Flan-PaLMChilla 62B was superior only for Level 1 extraversion.Source: §L.2 · p. 41
• Flan-PaLM 8B performed poorly across dimensions, producing only marginally distinguishable Level 1 and Level 9 distributions and appearing unsuitable for practical concurrent shaping.It nevertheless generated a non-trivial separation for extraversion.Source: §L.2 · pp. 41–42
• Openness was the hardest trait to shape concurrently, whereas extraversion was easiest, with Flan-PaLM 62B even outperforming Flan-PaLM 540B.The authors hypothesize that language correlations complicate openness, while broad and commonly understood extraversion language benefits smaller models’ in-context learning.Source: §L.2 · p. 42

M LLM Personality Traits in Real-World Task Methodology

• The real-world task evaluates whether shaping latent personality levels affects downstream behavior in open-ended, user-facing generation.The task was chosen to require incorporating personality-related information into writing rather than answering survey items.Source: M LLM Personality Traits in Real-World… · pp. 42–43

Personality Prediction API

• The Apply Magic Sauce API estimates personality from generated open-ended text using a method reported as more accurate than human ratings and less dependent on self-report.It was trained on a protected dataset absent from leading LLM pre-training corpora and on social-media status updates suited to the task.Source: Personality Prediction API · p. 43

Task Design

• Flagship models from each tested LLM family generated social-media status updates conditioned on psychodemographic profiles combining personality and demographic personas.The design was intended to focus inference on biographic and personality prompt content and produce socially elaborate text.Source: Task Design · p. 43
• Social-media updates were selected because they are autobiographical and rich in observable thoughts, emotions, and everyday behavior relevant to personality assessment.The task also differed from generic reading-comprehension and standard autobiographical writing tasks, reducing reliance on surface linguistic competence.Source: Task Design · p. 44
• The study reused 2,250 psychodemographic descriptions from the independent personality-shaping procedure and appended static instructions to generate status updates matching each description.This reuse allowed generated-text personality estimates to be statistically linked to IPIP-NEO scores from the same prompts.Source: Task Design · pp. 44–45
• The task covered work, family, friends, free time, romantic life, media consumption, and communication so personality could be rated across multiple social domains.The topic list was developed with psychometricians on the author team.Source: Task Design · pp. 45–46
• The generation scale ranged from 225,000 target updates for Flan-PaLM 540B to 56,250 generations and 1.125 million target updates per remaining non-Google model.For non-Google models, the robustness design requested 20 updates per prompt and repeated inference 25 times per prompt.Source: Task Design · p. 46

N LLM Personality Traits in Real-World Task Results

• Table 4 reports Spearman’s ρ between prompted personality levels and linguistic personality estimates from text generated under those prompts.Source: N LLM Personality Traits in Real-World… · p. 47
• The AMS API was used in a separate text-generation task to test whether psychometric test-based scores reflected personality.Source: N LLM Personality Traits in Real-World… · pp. 47–48
• Substantial correlations showed that LLM-simulated IPIP-NEO responses accurately captured latent personality signals manifested in downstream task behavior.Source: N LLM Personality Traits in Real-World… · p. 48
• Flan-PaLM 540B followed extreme personality prompts, generating social-media updates targeted at Level 1 or Level 9 of each Big Five dimension.Source: N LLM Personality Traits in Real-World… · pp. 48–49
• Word clouds showed stark differences between high- and low-trait generations, with most dominant terms relevant to the prompted personality dimension.Source: N LLM Personality Traits in Real-World… · p. 49
• Low agreeableness text contained more expletives, whereas high agreeableness text included more mentions of family members.Source: N LLM Personality Traits in Real-World… · p. 49
• Low neuroticism text included terms such as “relaxing” and “happy,” while high neuroticism text included extreme feeling-based words such as “hate” and “excited.”Source: N LLM Personality Traits in Real-World… · p. 50

O Discussion

• The discussion relates these findings to recent LLM performance trends along the axes of model training and scale.Source: O Discussion · p. 50

O.1 Effect of model post-training

• Instruction fine-tuning produced the strongest improvements in reliable, convergent, discriminant, and criterion-valid personality measurements, with smaller tuned models outperforming larger base models.Flan-PaLM 8B outperformed PaLM 62B, and Llama 2-Chat 7B outperformed Llama 2 70B; these comparisons are reported in Table 2 and Section 2.2.Source: §O.1 · p. 51
• Instruction fine-tuning generally improved internal consistency and composite reliability, but PaLM 62B variants had similarly high λ6 and ω despite differing α estimates.The authors explain this pattern through item difficulty and possible differences in the constructs measured by individual test items.Source: §O.1 · p. 51
• Compute-optimally trained Flan-PaLMChilla 62B outperformed Flan-PaLM 62B in independently shaping four Big Five domains, although their reliability and validity were otherwise indistinguishable.Overall, the findings indicate a positive association between training and the reliability and validity of synthetic personality measurements.Source: §O.1 · p. 51

O.2 Effect of model size

• Larger models showed stronger reliability, convergent validity, and criterion validity, including stronger correlations with theoretically related psychological constructs among larger instruction-tuned models.These effects were summarized in Table 2 and aligned with size-related improvements in reading comprehension, reasoning, and broad-context abilities.Source: §O.2 · p. 52

P Code Availability

• The psychometric-testing code and analysis code are interoperable across LLMs and available in an open-source repository for public use.The repository includes code for prompt generation, reliability and construct-validity analysis, and trait shaping.Source: P Code Availability · p. 52

Q Data Availability

• Generated psychometric scores and open-ended responses are publicly available, while test items, PersonaChat profiles, and the AMS API were used under their stated research or licensing conditions.The study sampled 50 biographic descriptions from PersonaChat and used AMS to predict personality from generated text.Source: Q Data Availability · p. 52

R Author Contributions

• The authors’ contributions covered project conception, psychometric methodology, infrastructure, personality shaping, downstream experiments, analysis, limitations, ethics, and manuscript preparation.The study was funded by Alphabet Inc. and/or a subsidiary, with several authors employed by Alphabet and affiliated with universities or supported by the Gates Cambridge Scholarship [OPP1144].Source: R Author Contributions · pp. 52–53, S Competing Interests · p. 53

S Competing Interests

• The study discloses Alphabet funding, Alphabet employment and possible stock ownership for several authors, and additional university affiliations and scholarship support.These disclosures identify the competing-interest relationships reported by the authors.Source: S Competing Interests · p. 53