PaperlayerView exact revision on arXiv

Source-linked AI summary

Controlling and Assessing Appropriate Persona Use in LLM-based Dialogue Generation

Jongkyung Shin, Inkyu Lee, Chiehyeon Lim

arXiv:2609.04676v1cs.CL

TL;DR

LLM-based PDG suffers from persona overuse, while existing metrics do not adequately assess contextually appropriate persona use. The paper analyzes these problems, introduces SCONPOS to suppress overuse through response-free representation intervention, and proposes PAS for appropriateness evaluation. Experiments show that SCONPOS reduces overuse and PAS captures contextual appropriateness.

  • Problem

    LLMs overuse persona attributes regardless of dialogue context, while no established metric adequately assesses persona-use appropriateness, including both overuse and underuse.

  • Method

    The paper analyzes persona use, introduces SCONPOS to suppress prompt-encoding bias in internal representations without response generation, and proposes PAS for contextual appropriateness.

  • Results

    SCONPOS systematically reduces persona overuse across datasets and models, while PAS captures the contextual appropriateness of persona use.

  • Takeaways & Limitations

    Appropriate persona use requires controlling when and to what extent attributes are reflected, rather than maximizing persona inclusion.

  • Takeaways & Limitations

    SCONPOS’s optimal suppression intensity and target layer may vary by model and dataset, requiring new configuration searches for new settings.

Abstract

from arXiv · show

In persona-based dialogue generation (PDG), LLMs often overuse persona attributes by incorporating them regardless of dialogue context, resulting in unnatural responses. Despite its practical significance, the underlying causes remain unexplored, with no method to mitigate this problem or metric to assess the appropriateness of persona use. To address these issues, we first conduct a comprehensive analysis of LLM-based PDG, revealing that LLMs exhibit a systematic bias to incorporate all given persona attributes, and that existing metrics fail to capture contextual appropriateness. Building on these findings, we propose Self-CONtrastive Persona Overuse Suppression (SCONPOS) to mitigate overuse by directly intervening in LLMs' internal representations at the prompt encoding stage, without requiring any response generation. We further propose the Persona Appropriateness Score (PAS), a novel metric that penalizes both overuse and underuse. Experimental results demonstrate that SCONPOS systematically reduces overuse, and PAS captures the contextual appropriateness of persona use.

1 Introduction

LLM-based PDG can produce contextually inappropriate persona use: models overuse supplied attributes, while existing evaluation does not adequately assess appropriateness. The paper analyzes these gaps and proposes SCONPOS and PAS to address them.

  • Motivation: PDG seeks responses that are fluent, contextually appropriate, and faithful to a predefined persona.It supports roles, traits, and identities across applications including counseling, games, and education.
  • Problem: Persona overuse causes LLMs to emphasize assigned attributes regardless of contextual relevance, disrupting dialogue flow and potentially undermining reliability and user trust.The problem is especially consequential in context-sensitive applications such as mental health counseling and customer service.
  • Problem: Persona underuse and overuse reflect the shared challenge of controlling when and to what extent persona attributes should be expressed.Underuse reduces persona consistency, while overuse produces contextually inappropriate responses.
  • Contributions: The analysis finds that LLMs incorporate all given persona attributes regardless of dialogue context, while existing metrics fail to capture contextual appropriateness.This identifies both a systematic model bias and an evaluation gap.
  • Contributions: SCONPOS mitigates persona overuse by suppressing overuse-inducing bias in LLM internal representations without requiring response generation.Experiments show effective and efficient reduction of persona overuse across datasets and models.
  • Contributions: PAS assesses the appropriateness of persona use by capturing how contextually appropriate persona attributes are.The paper presents PAS as a metric designed specifically for persona-use appropriateness.

2 Analysis

The analysis tests whether LLMs selectively use persona attributes according to dialogue context and whether existing metrics measure that appropriateness. Results show systematic persona overuse and inadequate evaluation of both overuse and underuse.

  • Research questions: The analysis asks whether LLMs control persona use based on the partner’s utterance and whether existing metrics assess persona-use appropriateness.These questions frame both model behavior and evaluation validity.
  • Analysis setup: The study compares All, Match, +Noise, and No settings to test whether models selectively reflect attributes aligned with the partner’s topic.All provides every attribute, Match provides topic-aligned attributes, +Noise adds one irrelevant attribute, and No provides none.
  • Analysis setup: The experiments use PersonaChat and MBTI-S2Conv across Llama3.1, Qwen2.5, DeepSeek-chat, and GPT4.1.The datasets differ in persona-attribute detail, enabling analysis across four models and two persona formats.
  • Results and implications: Overuse increased as more topic-irrelevant persona attributes were added, indicating that LLMs tend to incorporate supplied attributes rather than selectively using relevant ones.The same pattern appeared when both the agent’s and partner’s personas were provided.
  • Results and implications: Existing metrics were insufficient because consistency metrics failed to penalize overuse, Overuse failed to capture underuse, and reference-based metrics did not distinguish settings adequately.Consistency metrics favored All even for persona-irrelevant utterances, while Overuse favored No for persona-relevant utterances.

3 SCONPOS: Self-CONtrastive Persona Overuse Suppression

SCONPOS identifies overuse-inducing representations by contrasting prompts with all persona attributes against context-filtered prompts, then suppresses that bias during inference. The method extracts its steering signal from prompts alone and applies a single layer intervention without parameter updates.

  • Method overview: SCONPOS targets the overuse-inducing bias formed at the prompt encoding stage in the model’s internal representation space.Its goal is to make the model selectively reflect persona attributes appropriate to the current dialogue context.
  • Bias identification: Activation differences between All and Match settings identify representational components induced by context-irrelevant persona conditioning.These components are treated as the overuse-inducing bias to suppress.
  • Persona filtering: The method filters the full persona set using topics from the partner’s last utterance to construct a context-appropriate persona set.The resulting prompts form self-contrastive pairs with shared dialogue history.
  • Bias identification: The steering vector is extracted from prompt representations at the last prompt token of a selected layer in a frozen LLM.The vector is computed by averaging activation differences across the dataset.
  • Response-free intervention: SCONPOS extracts its steering signal from prompts without generating responses, unlike representation-engineering methods requiring response generation.This response-free design is a stated methodological distinction.
  • Inference intervention: During inference, SCONPOS suppresses the identified bias by subtracting a scaled steering vector at one target layer.The intervention uses suppression intensity λ and requires one subtraction operation without parameter updates or complex computations.

4 PAS: Persona Appropriateness Score

Persona Appropriateness Score (PAS) evaluates whether persona use matches the dialogue context by comparing topic distributions, while penalizing both irrelevant persona mentions and ignored relevant topics.

  • PAS measures the discrepancy between partner-utterance and response topic distributions to assess contextual appropriateness of persona use.Its topic space is persona-specific rather than a measure of general utterance–response coherence.
  • PAS introduces a dummy topic for utterances or responses that do not correspond to any predefined persona topic.
  • For each instance, PAS represents partner and response topic usage as discrete probability distributions within the persona-topic space.
  • PAS uses Jensen–Shannon Distance because it has metric properties and remains finite and stable when topic distributions do not overlap.
  • The overall PAS averages instance scores separately for persona-relevant and persona-irrelevant utterances, then combines them with a harmonic mean.This prevents models that never express persona attributes from receiving inflated scores on persona-irrelevant utterances.
  • PAS penalizes both overuse of irrelevant persona attributes and underuse of relevant persona topics.

5 Experiment

Experiments show that SCONPOS reduces persona overuse across datasets, models, settings, and evaluation methods, while PAS distinguishes contextually appropriate persona use from underuse and overuse.

  • Overuse mitigation: SCONPOS systematically reduces overuse across short-form and descriptive long-form personas, outperforming ICL-based methods and CAA.The comparison uses Llama3.1 in Table 2, with results for other models reported separately.
  • Transferability: SCONPOS transfers across datasets, reducing overuse and improving ROUGE-L and PAS, whereas CAA shows lower or unsuccessful transfer performance.The results support that SCONPOS captures overuse-inducing bias rather than dataset-specific surface patterns.
  • Controlled analysis: A controlled comparison finds comparable overuse reduction after duplicating relevant attributes, indicating the steering effect is not attributable to prompt length or attribute count.The evaluation uses persona-relevant utterances and controls both properties of the prompt.
  • Efficiency: SCONPOS is approximately 4.5× faster than CAA for extraction, uses 20KB of storage, and has comparable inference time.SCONPOS extracts vectors from prompts alone, avoiding response generation and the additional prompt-response extraction cost required by CAA.
  • Human evaluation: Human evaluation shows that SCONPOS consistently achieves higher win rates for persona appropriateness and dialogue coherence across ICL methods and datasets.The result confirms the mitigation effect in human judgments as well as automatic metrics.
  • Hyperparameter analysis: Increasing suppression intensity lowers Overuse, middle layers with ratios from 0.3 to 0.7 control persona use most effectively, and excessive intensity disrupts generation.Negative suppression intensity amplifies persona overuse, while excessively large values can produce abnormal activation values.
  • PAS evaluation: PAS assigns the highest score to Match and the lowest scores to No and All, thereby penalizing both persona underuse and overuse.The metric also assigns a lower score to All than +Noise, reflecting more severe persona overuse.
  • PAS evaluation: PAS shows a 48.4% difference between contextually appropriate and underuse cases, while Overuse, P-Dist, and ROUGE-L show only marginal differences.The comparison concerns No-setting responses for persona-relevant versus persona-irrelevant utterances.

6 Conclusion

The paper identifies persona overuse as an internal bias in LLM-based PDG and introduces SCONPOS and PAS to address mitigation and evaluation. Experiments show that SCONPOS reduces overuse, while PAS detects both overuse and underuse according to dialogue context.

  • Findings: The analysis finds that LLMs incorporate given persona attributes regardless of dialogue context, revealing a systematic persona-overuse bias.The conclusion characterizes this bias as fundamental rather than superficial.
  • Contributions: SCONPOS mitigates overuse by modifying internal representations at the prompt encoding stage without response generation.The conclusion states that the bias formed at prompt encoding influences the reasoning process as a whole.
  • Contributions: PAS detects both persona overuse and underuse according to dialogue context, unlike existing metrics.The paper presents this as a step toward more contextually appropriate persona-based dialogue systems.

Limitations

The paper identifies limitations in SCONPOS and PAS concerning configuration transfer, representation-level disentanglement, language coverage, and the scope of conversational context used to assess appropriateness.

  • Configuration dependence: SCONPOS’s optimal λ and target layer may vary across models and datasets, requiring new configuration searches for each application.The selected settings are not guaranteed to be globally optimal; automated or adaptive search is proposed as future work.
  • Representation disentanglement: The steering effect is supported as distinct from attribute count and prompt length, but fully disentangling it from correlated representation-space factors remains open.Control experiments and cross-dataset transfer provide evidence for the interpretation without completely resolving all correlations.
  • Language scope: Experiments and analyses use only English datasets, so generalization to other languages remains unclear.Persona expression and dialogue styles may differ across languages, potentially affecting observed overuse patterns and mitigation effects.
  • Evaluation scope: PAS assesses appropriateness through topical alignment with the partner’s last utterance, which may not capture natural topic shifts across the broader dialogue.The paper suggests extending evaluation to topic transitions throughout the full dialogue context.

Ethics Statement

The ethics statement highlights misuse risks from steering-based intervention and clarifies that PAS measures contextual persona alignment rather than ethical or social safety.

  • Potential misuse: SCONPOS could be misused by reversing its steering direction to encourage context-irrelevant persona attributes or excessive knowledge generation.Such misuse may increase risks involving sensitive information and harmful persona expression.
  • Metric scope: PAS should not be interpreted as an ethical or safety metric because contextual alignment does not guarantee that a response is socially appropriate.A malicious dialogue context could receive a high PAS for a response that aligns with a malicious persona.
  • Evaluation setup: The study evaluates PAS using topic annotation procedures that map persona attributes and utterances to predefined topics.GPT-4o is used to label persona attributes and assign topics, including implicit contextual meanings for partner utterances in PersonaChat.

A.2 Analysis Results with Both Agent and Partner Personas

With both agent and partner personas provided, LLMs still increase persona overuse as more attributes are supplied, while existing metrics continue to reward persona quantity over contextual appropriateness.

  • Observed overuse: Overuse increases as more persona attributes are included, even when both the agent’s and partner’s personas are provided.Additional persona information strengthens incorporation beyond what is appropriate for the partner’s current utterance.
  • Metric limitations: Existing metrics retain similar limitations in the both-persona setting and do not reliably capture contextual changes in persona use.Persona consistency metrics favor settings with more attributes, while reference-based metrics show only small differences.
  • Overuse metric: The analysis compares persona-topic counts between the partner’s utterance and the model response to quantify overuse.Topic-level scores use excess response counts and are averaged over the union of topics before sigmoid normalization.
  • Baseline metrics: C.Score measures entailment between each persona attribute and the generated response using a BERT-based natural language inference model.The fine-tuned bert-large model is trained on DNLI and assigns entailment, independence, or contradiction values.
  • Baseline metrics: P-Distance computes semantic similarity between persona keywords and response keywords, while P-F1 measures token-level overlap.P-Distance averages each persona keyword’s maximum similarity; P-F1 uses persona-response precision and recall.

B.3 Human Evaluation Settings

Human evaluation compares steered and unsteered responses for persona appropriateness and dialogue coherence, and separately validates PAS ratings using annotated samples.

  • Human evaluation: Ten fluent English undergraduate or graduate volunteers evaluated response pairs with and without steering.Comparisons covered Vanilla and ICL-based methods and used persona appropriateness and dialogue coherence rubrics.
  • Human evaluation: Each comparison set contained 50 response pairs judged by three annotators, with majority vote determining the final judgment and win rate reported.The evaluation targeted the superior response for persona appropriateness and dialogue coherence.
  • PAS validation: PAS validation used 500 PersonaChat samples rated for persona appropriateness on a 5-point Likert scale.The samples were distinct from the response-pair comparisons, and appropriateness and coherence rubrics were specified separately.
  • Configuration: SCONPOS and CAA used the same searched ranges for steering intensity λ and intervention target layer ratio.The search covered λ values from 1 to 5 and layer ratios from 0.1 to 1.0.
  • Decoding: Final evaluation used greedy decoding, whereas CAA used sampling only to generate responses required for steering-vector extraction.CAA extraction used All and Match responses with temperature 1.0 and top-p 1.0.

C.1 Additional Human Evaluation for PAS

The redesigned human evaluation used repeated ratings to compare PAS and other metrics with human judgments of persona appropriateness. PAS was the only metric consistently positively correlated across evaluation protocols, while annotator agreement remained low.

  • Human evaluation design: Three workers rated each of 100 response samples on a 3-point Likert scale, enabling sample-level aggregation and inter-annotator agreement measurement.Annotators met quality requirements including a HIT Approval Rate above 95%, more than 1,000 approved HITs, and residency in an English-speaking country.
  • Human evaluation design: Krippendorff’s α was 0.155, indicating low agreement and suggesting that persona appropriateness is subjective even among human annotators.
  • Metric alignment: PAS was the only metric showing consistently positive correlation with human appropriateness judgments across the original and redesigned protocols.
  • Metric alignment: Under the redesigned protocol, P-Dist correlated negatively with human judgments (r = −0.235, p = 0.019), while Overuse also correlated negatively (r = −0.197, p = 0.050).C.Score and P-F1 showed no discernible relationship with human judgments in either evaluation.

D.2 Evaluation Metric in PDG

PDG evaluation uses fluency and persona consistency perspectives, with automatic metrics measuring lexical overlap or semantic similarity to assess generated responses. The supplied material also identifies several evaluation and prompt artifacts used in the study.

  • Evaluation perspectives: PDG evaluation is broadly categorized into fluency and persona consistency.
  • Fluency metrics: BLEU and ROUGE-L typically evaluate fluency through lexical overlap between generated responses and human references.
  • Fluency metrics: Embedding-based metrics assess semantic similarity rather than relying solely on lexical overlap.
  • Evaluation materials: The study includes prompt figures for PersonaChat, MBTI-S2Conv, Chain-of-Thought, task decomposition, and self-refinement, alongside rubrics for persona appropriateness and dialogue coherence.
Loading 2609.04676v1…