PaperlayerView exact revision on arXiv

Source-linked AI summary

Estimating the Personality of White-Box Language Models

Saketh Reddy Karra, Son The Nguyen, Theja Tulabandhula

arXiv:2204.12000v2cs.CLcs.AI

TL;DR

The paper addresses the limited study of personality traits inherited by language models and develops methods to quantify those traits in models and their training datasets. It prompts models with a Big Five questionnaire, classifies responses with a Zero-shot classifier, and explores personality modification. Experiments find varied personality traits across models that reflect their training data, while alteration methods remain preliminary because personality changes may affect text generation.

  • Problem

    Studies of personality traits inherited by language models have been scarce, despite extensive work on biases in generated text.

    Source: §1 · pp. 2–3

  • Method

    The paper prompts white-box language models with a Big Five questionnaire and uses Zero-shot classification to quantify traits in model responses and training datasets.

    Source: §1 · p. 3, §4.2 · p. 11, §6 · p. 23

  • Results

    Experiments reveal varied personality traits across language models that reflect the datasets used in their training and influence generated outputs.

    Source: §5.2 · p. 17, §6 · p. 24

  • Takeaways & Limitations

    The work provides a starting point for assessing and understanding model personalities and explores finetuning-based approaches to alter them.

    Source: §4.3 · p. 12, §6 · p. 24

    Changing a model’s personality may affect its text-generation capabilities, and precise personality control requires further work.

    Source: §5.4 · p. 23, §6 · p. 24

Abstract

from arXiv · show

Technology for open-ended language generation, a key application of artificial intelligence, has advanced to a great extent in recent years. Large-scale language models, which are trained on large corpora of text, are being used in a wide range of applications everywhere, from virtual assistants to conversational bots. While these language models output fluent text, existing research shows that these models can and do capture human biases. Many of these biases, especially those that could potentially cause harm, are being well-investigated. On the other hand, studies that infer and change human personality traits inherited by these models have been scarce or non-existent. Our work seeks to address this gap by exploring the personality traits of several large-scale language models designed for open-ended text generation and the datasets used for training them. We build on the popular Big Five factors and develop robust methods that quantify the personality traits of these models and their underlying datasets. In particular, we trigger the models with a questionnaire designed for personality assessment and subsequently classify the text responses into quantifiable traits using a Zero-shot classifier. Our estimation scheme sheds light on an important anthropomorphic element found in such AI models and can help stakeholders decide how they should be applied as well as how society could perceive them. Additionally, we examined approaches to alter these personalities, adding to our understanding of how AI models can be adapted to specific contexts.

1. Introduction

This paper addresses the limited study of personality traits inherited by language models by quantifying and modifying traits in white-box models and their training datasets. It uses Big Five assessment, zero-shot classification, and experiments contrasting models with their corpora.

  • The paper quantifies personality traits in white-box language models and their underlying datasets, addressing a research gap alongside studies of model biases.It focuses on open-ended text generation and examines how personality can be modified for specific contexts.Source: §1 · pp. 2–3
  • Identifying model personalities may inform application selection, bias analysis, fairness, and the design of more engaging or context-appropriate AI systems.Source: §1 · p. 2
  • The proposed estimation scheme prompts models with the Big Five questionnaire and classifies generated responses into quantifiable traits using Zero-shot learning (ZSL) (Yin et al., 2019).The same framework is applied to both datasets and model outputs.Source: §1 · p. 3
  • The paper examines personality modification through finetuning and a pretrained auxiliary personality classifier.Source: §1 · p. 3
  • Experiments compare multiple white-box language models with their training corpora, finding differing personality profiles and insights into factors associated with differences across models and data.The experiments are reported as validating the proposed methodology.Source: §1 · p. 4

2. Related Work

Related work covers human personality measurement, automatic personality detection, and societal biases in language modeling. The paper positions its ZSL-based Big Five estimation as a complementary approach focused on personality traits in language models.

2.1. Personality Measures

  • The Big Five factors are adapted from established personality measures to quantify traits of white-box language models and their training datasets.Related measures include 16PF, EPQ-R, MBTI, and PEN.Source: §2.1 · p. 4

2.2. Methods for Automatic Personality Detection

  • Prior automatic personality detection uses linguistic features, deep learning, regression, social-media analysis, and multimodal methods to infer Big Five traits from text.Source: §2.2 · p. 5
  • The paper uses Zero-shot learning to address personality classification without relying on large personality-annotated datasets, extending prior ZSL-based detection work to language models.Source: §2.2 · p. 5

2.3. Study of Biases in Language Modeling

  • The paper distinguishes its focus on learned personality traits from prior studies of societal biases in language modeling.It characterizes this focus as narrow, well-defined, and differentiating from earlier bias work.Source: §2.3 · pp. 5–6

3. Preliminaries

The preliminaries define the Big Five framework, motivate estimating model personality from generated responses, introduce Zero-shot classification, and describe the pretrained language models studied.

3.1. Big Five factors

  • The Big Five framework measures Extraversion, Neuroticism, Agreeableness, Conscientiousness, and Openness, with Emotional Stability estimated instead of Neuroticism.The questionnaire contains fifty statements scored from 1 to 5 and averages factor scores into quantifiable traits.Source: §3.1 · p. 6

3.2. Our Hypothesis for Personality Estimation

  • The paper hypothesizes that prompted language models generate responses reflecting personality traits of their training datasets, which auxiliary prediction models can quantify.Source: §3.2 · p. 7

3.3. Zero-shot Classifier

  • Zero-shot classification treats the input as a premise, constructs trait hypotheses, and uses an NLI model’s entailment and contradiction probabilities to estimate trait levels.The method is motivated by limited personality-annotated data and avoids requiring prior task-specific training data.Source: §3.3 · pp. 7–8

3.4. Language Models in Open Ended Text Generation

  • The study examines multiple pretrained autoregressive Transformer language models differing in training strategies and corpora, including GPT-2, GPT-3, GPT-3.5-Turbo, and LLaMA.Their training data include WebText, Common Crawl, Wikipedia, C4, Github, Gutenberg, Books3, ArXiv, and Stack Exchange.Source: §3.4 · pp. 8–9

4. Methods for Estimating and Altering Personalities

The paper presents a ZSC-based framework for estimating Big Five personality traits in datasets and language models, then describes two model-agnostic ways to modify those traits.

4.1. Evaluating Personalities of Datasets

  • Dataset personality estimation converts trait labels into entailment hypotheses and maps ZSC probabilities to Big Five scores using alternative single-label or extreme-end approaches.Approach 1 uses linear interpolation; Approach 2 uses extreme-end labels and softmax; Approach 3 evaluates both extremes dependently.Source: §4.1 · pp. 10–11
  • The third ZSC approach is expected to provide more precise dataset personality assessments by jointly evaluating both extremes of each trait and distinguishing negative relation from neutrality.The first approach can misread unrelated hypotheses, while the second omits the NLI model’s neutral score.Source: §4.1 · pp. 10–11

4.2. Evaluating Personalities of Language Models

  • Language-model evaluation repeatedly prompts each questionnaire statement, classifies independent completions with Approach 3, and summarizes interpolated scores across responses.Repeated prompting accounts for stochastic outputs, while independent statements avoid order effects.Source: §4.2 · pp. 11–12, Figure 1 · p. 12

4.3. Modifying Personalities of Language Models

  • Method 1 fine-tunes models on personality-annotated text for straightforward trait adaptation, whereas Method 2 fine-tunes an auxiliary binary classifier before text generation.Method 1 offers limited precision, and maintaining generation quality while targeting exact trait values is non-trivial.Source: §4.3 · pp. 12–13

5. Experiments

The experiments quantify personality traits in datasets and language models, compare their distributions, and test whether finetuning can alter model traits. Results show dataset–model similarity depends on prompting, model personalities differ, and trait alteration is possible but not precisely isolated.

5.1. Traits of Datasets

  • Dataset evaluation used sentence or small-paragraph samples because longer documents produced mostly neutral scores when they combined conflicting or traitless content.The resulting sample-level scores were aggregated into personality-trait distributions.Source: §5.1 · p. 14
  • Dataset trait distributions were generally positively skewed above the neutral score of 3.0, with Agreeableness and Extraversion often prominent and spreads differing by corpus.BookCorpus showed prominent Extraversion and Agreeableness, while Wikitext103 was more neutral and had the widest Agreeableness spread.Source: §5.1 · pp. 14, 16, Figure 4 · p. 15, Table 3 · p. 15

5.2. Traits of Models

  • Models were prompted with the Big Five questionnaire 30 times and classified with the ZSC framework to account for stochastic responses.GPT-3 and GPT-3.5-Turbo were evaluated through the OpenAI API, while GPT-2, TransformerXL, and XLNet were evaluated as white-box models.Source: §5.2 · p. 16
  • Language models exhibited distinct personality profiles: GPT-3.5-Turbo scored highest in Agreeableness and Extraversion, while LLaMA scored highest in Conscientiousness and Emotional Stability.These differences were reported from median trait scores and illustrated in the model-distribution figures.Source: §5.2 · p. 17, Table 5 · p. 17, Figure 5 · p. 18
  • GPT-2’s estimated traits varied across evaluation modes, with Agreeableness least sensitive and Openness most variable according to Wasserstein-distance comparisons.GPT-3, XLNet, and TransformerXL showed unchanged distributions because their responses were single sentences.Source: §5.2 · p. 19, Table 6 · p. 19, Figure 7 · p. 20

5.3. Traits of Datasets vs Models

  • Prompted GPT-2 trait distributions differed significantly from those of WebText, whereas unprompted random generation produced nearly identical distributions, making questionnaire prompting more reliable for model traits.The authors attribute the unprompted limitation to WebText’s predominantly factual content.Source: §5.3 · pp. 19, 21, Table 7 · p. 21

5.4. Altering the Traits of Models

  • Finetuning GPT-2 changed Agreeableness, Conscientiousness, Emotional Stability, and Openness, but also altered non-target traits, limiting precise control.Method 2 similarly improved Extraversion while noticeably changing other personality scores.Source: §5.4 · pp. 22–23, Table 8 · p. 23, Table 9 · p. 23

6. Conclusion

The paper quantifies personality traits in datasets and open-ended language models, then explores altering model personalities through personality-annotated fine-tuning. Its experiments find varied model traits reflecting training datasets, while precise personality alteration remains a future goal.

  • The paper quantifies personality profiles of datasets and language models using a pre-trained zero-shot classifier and Big Five questionnaire prompts.The approach extends dataset personality estimation to model responses elicited with the Big Five personality test questionnaire.Source: §6 · pp. 23–24
  • Language models possess varied personality traits that reflect their training datasets and influence their generated outputs.Source: §6 · p. 24
  • The paper proposes fine-tuning white-box language models on personality-annotated datasets as a straightforward way to alter personality traits.Source: §6 · p. 24
  • Future work aims to develop robust approaches that can precisely alter language-model personality traits and incorporate additional assessments such as the Myers-Briggs Type Indicator.Source: §6 · p. 24

Appendix A. Additional Details

The appendix defines the Wasserstein distance and training concepts, and documents additional fine-tuning results and questionnaire materials. It also identifies the optimizer and regularization terminology used in the experiments.

  • Wasserstein distance measures the work required to transform one probability distribution into another, with the infimum taken over variable pairs having distributions P and Q.Source: Appendix A · p. 28
  • Adam extends stochastic gradient descent with adaptive learning rates for each parameter, supporting faster and more accurate convergence under noisy or sparse gradients.Source: Appendix A · p. 28
  • Weight decay is a regularization hyperparameter that adds a loss penalty to encourage simpler and smoother model weight configurations.Source: Appendix A · p. 28
  • The appendix provides detailed Method 2 fine-tuning results for GPT-2 using conscientiousness, openness, emotional stability, and agreeableness labels.Tables A.10–A.13 report scores before and after fine-tuning at different thresholds.Source: Appendix A · p. 28, Table A.10 · p. 28, Table A.11 · p. 29
  • The appendix includes the questions used in the Big Five questionnaire.Source: Table A.14 · p. 30
Loading 2204.12000v2…