PaperlayerView exact revision on arXiv

Source-linked AI summary

FATE-5: Faithful Alignment of Trait Expression for Continuous Big Five Personality Control

Gunhee Cho, Hongyeob Kim, Yun-Gyung Cheong

arXiv:2610.34889v2cs.CL

TL;DR

Existing personality-control methods face a tradeoff between strong trait expression and natural conversation, despite the value of adapting conversational agents to different users and settings. FATE-5 separates continuous Big Five personality encoding from preference alignment for naturalness. Across three backbones, it maintains strong trait expression while producing substantially more natural dialogue than other trait-strong approaches.

  • Problem

    Existing approaches face a tradeoff between strong personality induction and natural conversation, limiting the joint goal of faithfully and naturally expressing a target profile.

    Source: §1 · p. 2, §3.1 · p. 4

  • Method

    FATE-5 learns soft prompts for continuous Big Five profiles with dialogue and survey supervision, then uses preference alignment based on dialogue quality and personality agreement.

    Source: §1 · p. 2

  • Results

    Across three backbones and human evaluation, FATE-5 maintains strong trait expression while producing substantially more natural dialogue than other trait-strong approaches.

    Source: §6 · p. 9

  • Takeaways & Limitations

    The findings suggest that personality fidelity and natural dialogue quality can be jointly maintained in personality-controllable conversational LLMs.

    Source: §6 · p. 9

Abstract

from arXiv · show

Controlling personality in large language models is important for building conversational agents that can be customized to different users and interaction settings. However, existing personality control methods often face a tradeoff between strong trait expression and natural dialogue generation. We introduce FATE-5, Faithful Alignment of Trait Expression for the Big Five, a two-stage framework that separates personality encoding from naturalness alignment. FATE-5 first learns continuous soft prompt representations of Big Five profiles using dialogue supervision and an auxiliary survey objective. It then improves dialogue naturalness through preference alignment based on both dialogue quality and personality agreement. Across three language model backbones, FATE-5 maintains strong trait expression while achieving substantially higher dialogue naturalness than other trait-strong methods. On held-out questionnaires, it achieves monotonicity correlations of 0.97-0.99 and recovery RMSE of 8.9-12.4, substantially improving over the evaluated baselines. These results demonstrate effective continuous personality control while maintaining natural conversational behavior.

1 INTRODUCTION

The introduction frames continuous Big Five control as useful for adapting conversational agents, but existing approaches struggle to balance strong trait expression with natural dialogue. FATE-5 separates personality encoding from naturalness alignment and evaluates both expression and control fidelity.

1 INTRODUCTION

  • Existing personality-control methods trade strong trait expression against natural dialogue, leaving continuous control insufficiently resolved.The Big Five offers five continuous dimensions for specifying a trait configuration and intensity.Source: §1 · p. 1
  • The paper characterizes continuous Big Five control along two dimensions—trait expression and dialogue quality—using multi-turn dialogue evaluation.It evaluates perceptible trait expression and recovery of intended profiles with complementary interactive dialogue and psychometric protocols.Source: §1 · p. 2
  • FATE-5 learns continuous soft-prompt representations with dialogue and survey supervision, then preference-aligns dialogue quality and personality agreement.The second stage freezes the encoder and backbone and trains a LoRA adapter using Direct Preference Optimization.Source: §1 · p. 2
  • FATE-5 is evaluated across multiple backbones and questionnaire formats, with ablations of key components assessing expression and control fidelity.The evaluation uses both dialogue and psychometric protocols.Source: §1 · p. 2

2 RELATED WORK

Prior work induces Big Five traits through prompting, parameter or representation changes, and activation steering; the related work also reviews psychometric instruments and evaluation sensitivities.

2.1 PERSONALITY CONTROL IN LARGE LANGUAGE MODELS

  • Prompt-based methods are inexpensive and model agnostic but often induce weaker trait expression than direct interventions, while representation and activation methods provide stronger control with less clear conversational quality.The paper reports that stronger trait control can reduce dialogue naturalness and positions FATE-5 as addressing that tradeoff.Source: §2.1 · p. 2

2.2 PSYCHOMETRIC EVALUATION OF LLM PERSONALITY

  • Psychometric evaluation of LLM personality uses established inventories and benchmarks, including the BFI, NEO-FFI, IPIP NEO 120, TIPI, BFI-2, and TRAIT.Prior work also compares questionnaire response formats and finds they can affect estimated LLM personality.Source: §2.2 · p. 3
  • Because measurements vary with prompt wording and response format, and questionnaire items may occur in pretraining data, the paper uses separate training and evaluation inventories and adds dialogue evaluation.Source: §2.2 · p. 3

3 METHOD

FATE-5 targets dialogue that expresses a continuous Big Five profile naturally. It separates profile encoding from preference alignment intended to improve naturalness while preserving trait expression.

3.1 PROBLEM SETUP

  • The goal is dialogue that makes target Big Five levels perceptible while remaining natural and engaging.Source: §3.1 · p. 4

3.2 PERSONALITY ENCODER

  • A lightweight encoder maps each continuous Big Five profile to soft-prompt embeddings prepended to input, training only the encoder while freezing the backbone.The encoder is a two-layer MLP with GELU activations.Source: §3.2 · p. 4

3.3 TRAINING OBJECTIVE: DIALOGUE AND SURVEY

  • Dialogue supervision trains responses from profile-context-reference triples, while questionnaire pseudo-answer supervision strengthens how faithfully the soft prompt preserves target trait values.The auxiliary survey loss trains the conditioned model to produce questionnaire answers derived from the target profile.Source: §3.3 · p. 4

3.4 TRAIT-PRESERVING NATURALNESS ALIGNMENT VIA OFFLINE DPO

  • Stage 2 restores naturalness through offline preference alignment while preserving the encoded profile, updating a LoRA adapter against a reference policy with a trait-aware reward.The encoder and backbone remain frozen during DPO optimization.Source: §3.4 · p. 5

4 EXPERIMENTS

The experiments combine complementary dialogue sources and frozen-backbone methods, then assess dialogue expression, questionnaire-based control, and component ablations. Evaluation spans three backbones and includes held-out questionnaire formats.

4.1 DATA

  • About 46k training examples combine coarse, single-trait Big5Chat dialogues with real Reddit conversations labeled by continuous Big Five scores.The mixture contains roughly 30k Big5Chat and 16k PANDORA examples; held-out TIPI and BFI-2 are reserved for evaluation.Source: §4.1 · p. 5

4.2 MODELS AND BASELINES

  • Experiments use three instruction-tuned backbones with frozen weights and compare prompting, steering vectors, PsychAdapter, and DPG against FATE-5.The trained baselines use the same personality dialogue corpus but omit the survey loss.Source: §4.2 · p. 6

4.3 EVALUATION

  • Evaluation measures both target-personality expression in free dialogue and fidelity to standardized questionnaire scores, alongside ablations.The protocols assess complementary dimensions and address the paper's research questions.Source: §4.3 · p. 6

4.3.1 DIALOGUE EXPRESSION (RQ1).

  • Dialogue expression is tested in topic-targeted 10-turn conversations using trait accuracy and 0–100 naturalness ratings from LLM judges and human raters.The human study collects 984 dialogue-level evaluations; aggregate split-half reliability reaches 0.90 for Trait and 0.94 for Naturalness.Source: §4.3.1 · pp. 6–7, Table 1 · p. 7

4.3.3 ABLATION OF OUR METHOD (RQ4).

  • Llama-3-8B ablations compare encoder and DPO combinations, remove survey supervision or the reward's trait term, and replace DPO with SFT.The comparisons test learned versus numeric-prompt conditioning and the roles of supervision and preference alignment.Source: §4.3.3 · p. 7

4.3.2 QUESTIONNAIRE MEASUREMENT (RQ2 AND RQ3).

  • Held-out questionnaires measure directional control by target–recovered-score correlation and absolute profile agreement by RMSE under natural phrasing.Monotonicity is also tested with rigid single-digit responses, across three backbones.Source: §4.3.2 · p. 7, Table 2 · p. 8

5 RESULTS

Across three backbones, FATE-5 combines strong trait expression with higher dialogue naturalness than trait-strong activation methods. Held-out questionnaire tests show strong monotonicity and lower profile-recovery error, while ablations support the roles of its components.

5.1 RQ1: EXPRESSION

  • 72.7–76.3 trait accuracy and 43.0, 39.1, and 45.4 naturalness: FATE-5 maintains trait expression while outperforming activation-based approaches on naturalness across three backbones.Steering naturalness is 2.6, 7.9, and 21.6 across Llama-3-8B, Gemma-2-9B, and OLMo-2-13B.Source: §5.1 · p. 8

5.2 RQ2: CONTINUOUS CONTROL

  • 0.99 under both formats on Llama-3-8B and Gemma-2-9B, and 0.97 and 0.98 on OLMo-2-13B: FATE-5 shows strong monotonicity across backbones.Prompting's monotonicity drops to 0.65 under natural phrasing on OLMo-2-13B.Source: §5.2 · p. 8

5.3 RQ3: QUESTIONNAIRE GENERALIZATION

  • 10.5, 8.9, and 12.4 RMSE on Llama-3-8B, Gemma-2-9B, and OLMo-2-13B: FATE-5 has the lowest error in every setting.Prompting's errors are 20.7, 18.0, and 25.4; injection-based baselines generally remain close to or above the 28.9 no-control reference. Overall, FATE-5 recovers profiles closer to target values across models.Source: §5.3 · pp. 8–9

5.4 RQ4: ABLATION

  • Ablations on Llama-3-8B support complementary encoder and DPO roles, while survey supervision and a trait-weighted reward support expression and alignment.Removing the encoder under DPO lowers trait accuracy from 75.3 to 60.0, while removing DPO lowers naturalness from 43.0 to 37.5. Removing survey loss lowers trait accuracy from 75.3 to 51.3; λ=2 raises trait accuracy to 78.7 but reduces naturalness to 32.5.Source: Table 3 · p. 9, §5.4 · p. 9

6 CONCLUSION

The conclusion presents FATE-5 as separating personality representation from naturalness alignment, with experiments supporting strong expression, natural dialogue, and improved questionnaire-based control.

6 CONCLUSION

  • Across three backbones and human evaluation, FATE-5 maintains strong trait expression while producing more natural dialogue than other trait-strong approaches.Psychometric evaluation shows reliable continuous control and lower profile recovery error; ablations support survey supervision and preference alignment.Source: §6 · p. 9

AI USE STATEMENT

The authors report using generative AI for parts of the experimental process and manuscript preparation, with authors reviewing and verifying the work.

AI USE STATEMENT

  • Generative AI assisted with code development, debugging, manuscript writing, and language refinement; the authors say they reviewed and verified AI-assisted work.The authors retain responsibility for the final content, including experimental results and claims.Source: AI USE STATEMENT · p. 10

ETHICS STATEMENT

The section documents participant protections and reproducibility practices, then details deterministic survey pseudo-answers and how they support training without exposing evaluation inventories or formats.

ETHICS STATEMENT

  • Human evaluation received prior Institutional Review Board approval, with voluntary informed consent and no personally identifiable information reported.Responses were used only for this research and collected through Amazon Mechanical Turk.Source: ETHICS STATEMENT · p. 10

REPRODUCIBILITY STATEMENT

  • The paper reports model, dataset, baseline, protocol, and training details across the main text and appendices, with fixed random seeds.Model and optimization configurations are reported for each training stage.Source: REPRODUCIBILITY STATEMENT · p. 10

A PSEUDO-ANSWER CONSTRUCTION FOR THE SURVEY LOSS

  • Survey pseudo-answers are derived by inverting the scoring rubric, mapping each target trait to the nearest questionnaire response option.Positive- and reverse-keyed items encode agreement and disagreement respectively; applying the rubric recovers the target up to rounding.Source: A PSEUDO-ANSWER CONSTRUCTION FOR THE… · pp. 12–13
  • Survey targets comprise 200 independently sampled profiles, with each trait uniform on 1–5 and converted to the encoder’s 0–100 scale.The survey objective uses training inventories, while evaluation inventories and conversational phrasing are withheld from training.Source: A PSEUDO-ANSWER CONSTRUCTION FOR THE… · pp. 12–13
  • Training rotates through training questionnaires, samples eight items per target, and enables the survey loss after a 300-step dialogue-only warmup.The combined losses use λdial=0.6 and λsurv=0.4; only the answer digit contributes to the survey loss.Source: A PSEUDO-ANSWER CONSTRUCTION FOR THE… · p. 13

B TRAINING AND REPRODUCTION DETAILS

The experiments use frozen backbones and matched training conditions for baselines, with separate encoder and preference-alignment stages and shared dialogue-generation settings.

B TRAINING AND REPRODUCTION DETAILS

  • All methods are trained and evaluated separately per backbone, with frozen backbone weights, bfloat16, and seed 42.Source: B TRAINING AND REPRODUCTION DETAILS · p. 13
  • The personality encoder is a two-layer, 512-hidden-unit MLP that produces 64 soft-prompt tokens and trains for 3,000 steps.It uses AdamW with learning rate 10^-3, weight decay 0.01, and an effective batch size of eight.Source: B TRAINING AND REPRODUCTION DETAILS · p. 13
  • Preference alignment samples five candidates per profile-topic pair and selects highest- and lowest-reward outputs, discarding zero-gap pairs.It uses 200 random profiles, ten dialogue topics, and Llama-3.1-70B-Instruct as the scoring judge.Source: B TRAINING AND REPRODUCTION DETAILS · p. 13
  • DPO trains a rank-16 LoRA adapter for 200 optimizer steps while keeping the encoder and backbone frozen.All methods share dialogue decoding settings, including temperature 0.7, top-p 0.9, and a maximum of 120 new tokens per turn.Source: B TRAINING AND REPRODUCTION DETAILS · pp. 13–14
  • Baseline reimplementations use the same corpus, 3,000-step budget, optimizer, effective batch size, sequence length, and MLP size to focus comparison on conditioning mechanisms.Learning rate and injection scale are selected on a held-out training subset using validation loss, excluding degenerate text settings.Source: B TRAINING AND REPRODUCTION DETAILS · p. 14
  • The trained baselines use distinct conditioning mechanisms: layerwise additive vectors, hypernetwork-generated rank-8 LoRA modulation, and activation steering.The numeric Big5 Scaler baseline uses the authors’ published prompt format without additional training.Source: B TRAINING AND REPRODUCTION DETAILS · p. 14

C HUMAN EVALUATION SETTING

The human evaluation compares blinded dialogues across methods using trait and naturalness ratings, and the following ablation section specifies a numeric Big Five prompt.

C HUMAN EVALUATION SETTING

  • 50 dialogues use ten shared persona-topic configurations, with one dialogue selected per topic for each of five methods.Source: C HUMAN EVALUATION SETTING · p. 14
  • Each dialogue is rated for trait expression and naturalness, with method identities hidden and trait prompts phrased as behavioral descriptions.Naturalness uses a 1–5 scale rescaled to 0–100; trait judgments include an “I don’t know” option.Source: C HUMAN EVALUATION SETTING · p. 14
  • 59 workers provided 984 dialogue-level evaluations; “I don’t know” responses made up 5.2% of trait judgments and were excluded from trait accuracy.Each worker evaluated one form, and no worker rated the same dialogue more than once.Source: C HUMAN EVALUATION SETTING · p. 14
  • Inter-rater alpha was 0.49 for Trait and 0.45 for Naturalness, while split-half reliability reached 0.90 and 0.94 respectively.Human and LLM naturalness scores correlated at Pearson r=0.84 and Spearman ρ=0.85 across 50 dialogues.Source: C HUMAN EVALUATION SETTING · p. 15

D NUMERIC PROMPT USED IN THE ABLATION

  • The numeric-prompt ablation specifies a profile with Openness 75, Conscientiousness 20, Extraversion 60, Agreeableness 45, and Neuroticism 30, each out of 100.The prompt asks the model to express those traits naturally in conversation.Source: D NUMERIC PROMPT USED IN THE ABLATION · p. 15

E WHY REMOVING THE TRAIT ANCHOR ALSO HURTS NATURALNESS

Removing the personality component leaves many preference candidates tied and sharply reduces usable training pairs; naturalness also drops as a result of the preference data.

E WHY REMOVING THE TRAIT ANCHOR ALSO HURTS NATURALNESS

Loading 2610.34889v2…