Source-linked AI summary
Two Tales of Persona in LLMs: A Survey of Role-Playing and Personalization
Yu-Min Tseng, Yu-Chao Huang, Teng-Yun Hsiao, Wei-Lin Chen, Chao-Wei Huang, Yu Meng, Yun-Nung Chen
TL;DR
Persona research for LLMs is growing but remains disorganized and lacks a systematic taxonomy. This survey unifies the field through a comprehensive categorization of role-playing and personalization, adds personality-evaluation methods, and identifies challenges including limited benchmarks and bias.
Problem
Research on leveraging personas in LLMs lacks a unifying overview and systematic categorization.
Method
The paper surveys and categorizes LLM role-playing, LLM personalization, personality evaluation, and future research directions.
Results
The survey distinguishes role-playing, where personas belong to LLMs, from personalization, where LLMs use user personas, and presents a unified persona view.
Takeaways & Limitations
The taxonomy and maintained paper collection provide an introductory guide and practical roadmap for research on personas in LLMs.
Takeaways & Limitations
The paper omits a comprehensive evaluation comparison because studies use diverse subtasks and metrics, and some scenarios require multiple metrics.
Abstract
from arXiv · showhide
The concept of persona, originally adopted in dialogue literature, has re-surged as a promising framework for tailoring large language models (LLMs) to specific context (e.g., personalized search, LLM-as-a-judge). However, the growing research on leveraging persona in LLMs is relatively disorganized and lacks a systematic taxonomy. To close the gap, we present a comprehensive survey to categorize the current state of the field. We identify two lines of research, namely (1) LLM Role-Playing, where personas are assigned to LLMs, and (2) LLM Personalization, where LLMs take care of user personas. Additionally, we introduce existing methods for LLM personality evaluation. To the best of our knowledge, we present the first survey for role-playing and personalization in LLMs under the unified view of persona. We continuously maintain a paper collection to foster future endeavors: https://github.com/MiuLab/PersonaLLM-Survey
1 Introduction
Persona research adapts LLMs to specific contexts, but the literature lacks a unifying overview. This survey organizes the field into role-playing and personalization and adds evaluation methods and research guidance.
- Persona-based adaptation helps LLMs generate contextually appropriate responses for specific applications.
- Role-playing assigns personas to LLMs, whereas personalization has LLMs use user personas to meet individualized needs.
- The survey provides a comprehensive review and systematic categorization of existing persona studies.
- It presents the first survey unifying LLM role-playing and personalization under the concept of persona.
- The taxonomy covers role-playing, personalization, personality evaluation, and future challenges to guide researchers toward relevant tasks and scenarios.
2 LLM Role-Playing
LLM role-playing equips language agents with assigned personas and environmental feedback across software, game, medical, and evaluation settings. The survey distinguishes single-agent from multi-agent schemas and discusses collaboration, emergent behavior, and safety concerns.
- Role-Playing Environments: Role-playing commonly couples personas with language agents through prompts, providing a simple training-free adaptation paradigm.
- Role-Playing Environments: Role-playing environments include software development, games, medical applications, and LLM-based evaluation.
- Game: Game environments often rely on retrieval-based memory streams because agents must respond to extensive settings, tools, and situations.
- Medical Application: Medical role-playing assigns general or specialist personas, and surveyed studies report performance enhancement through this adaptation.
- Role-Playing Schema: The taxonomy separates single-agent schemas, where agents act independently, from multi-agent schemas, where cooperation or communication is necessary.
- Emergent Behaviors: Multi-agent role-playing can produce conformity and consensus-related behaviors, while destructive behavior raises safety and bias concerns.
3 LLM Personalization
LLM personalization uses user information and behavior to tailor outputs across recommendation, search, education, healthcare, and dialogue. The surveyed methods span prompting, fine-tuning, memory, retrieval, and persona-aware response generation.
- Personalized LLMs use individual information and historical behaviors to satisfy customized user needs.
- Recommendation: Recommendation research explores personal-information incorporation, aspect-based prompting, personalized chit-chat, fine-tuning, and zero-shot pipelines.
- Search: Personalized search combines complex-query understanding and interaction history with memory retrieval and natural-language information synthesis.
- Education: Educational applications include step-by-step explanations, technical question answering, lecture summarization, learning assistance, assessment, and feedback.
- Healthcare: Personalized healthcare frameworks integrate external data with users’ personalized health data to address medical problems.
- Dialogue: Dialogue personalization includes task-oriented modeling and user persona modeling, with the latter detecting personas from dialogue history to generate customized responses.
4 LLM Personality Evaluation
The survey reviews methods for evaluating whether adapted LLMs express their intended personas. Existing approaches use human personality assessments, personality inventories, and interviews, but their transferability to LLMs remains unresolved.
- Evaluation asks whether role-playing and personalized LLMs accurately reflect their intended personas after adaptation.
- Human assessments commonly use the Big Five and MBTI frameworks to measure LLM personality traits.
- Machine Personality Inventory evaluates Big Five traits and compares internal consistency with human evaluation and model capabilities.
- Interview-based personality testing finds that demonstrated personalities align well with assigned character personas.
- Whether human psychometric tests transfer directly to LLM evaluation remains an open question.
5 Challenges and Future Directions
The survey identifies limitations in current persona research, including task dependence, data and benchmark scarcity, bias, safety risks, and privacy concerns. It points toward more general frameworks, richer protected datasets, broader evaluations, and further mitigation work.
- Towards a General Framework: Role-playing frameworks are mostly task dependent and rely heavily on human-crafted personas, requiring task knowledge and understanding.
- Towards a General Framework: Automatic persona identification and dynamic adjustment improve flexibility, but existing approaches may still require limited roles, human assignments, prior knowledge, or predefined configurations.
- Efficient Personalization: Personalization can exceed context length and increase inference costs when user histories are included, while retrieval may lose information.
- Lack of Datasets and Benchmarks: Datasets and benchmarks remain limited across role-playing and personalization, especially for specialized environments, contradictory or multimodal personas, and comprehensive evaluation dimensions.
- Lack of Datasets and Benchmarks: Privacy concerns create a scarcity of high-quality personal data, hindering thorough evaluation of personalization methods.
- Bias: Role-playing and personalization can induce stereotypical, harmful, popularity-based, position-based, and agree biases.
- Safety: Studies report collaborative jailbreaks, persona-assisted jailbreaking, negative role-playing behaviors, and toxicity across topics.
- Privacy: Personalization raises privacy risks because membership inference attacks can leak encoded personal information.
6 Broader Implications
The survey highlights potential benefits of personalization in education alongside risks in unequal access and future healthcare deployment. It also emphasizes that LLM personality measurement lacks a unified framework and consistency remains imperfect.
- Education and Healthcare: Personalized education could provide accessible content, lecture materials, and affordable tutoring, benefiting minority groups with limited resources.
- Education and Healthcare: Educational personalization may also intensify polarization if privileged groups receive private tutors while underrepresented groups receive only LLM-powered support.
- Education and Healthcare: Potential healthcare uses, including clinical scenarios, mental health assessments, and therapeutic treatments, raise unresolved liability and legal questions.
- Personality Evaluation: Personality evaluation lacks a unifying quantification framework, and LLMs sometimes fail to maintain consistent personalities.
7 Conclusion
The survey organizes persona research into role-playing, personalization, and personality evaluation while highlighting challenges and future directions. It provides a maintained resource intended as an introductory guide and practical roadmap.
- The survey summarizes role-playing and personalization research, presents LLM personality evaluation methods, and identifies challenges and future directions.
- The authors position the survey and its resources as an introductory guide for beginners and a practical roadmap for future research.
Limitations
The survey does not provide a comprehensive evaluation comparison because studies use heterogeneous tasks and metrics, while some scenarios require multiple metrics.
- The survey omits a comprehensive evaluation comparison because studies address different subtasks and use varied or self-defined metrics.Examples include persona accuracy, task success rate, and combined inform and success rate.
A Web
The survey reviews web-environment research on autonomous navigation, HTML understanding, visual grounding, and benchmark design, alongside datasets and task categories for personalized recommendation and dialogue modeling.
- Web environments: Web agents autonomously navigate webpages by clicking items, capturing contents, and searching external knowledge, with HTML understanding and visual grounding as key components.The cited benchmark comparison distinguishes realistic environments, dynamic interaction, visual requirements, and end-to-end versus fine-grained assessment.
- Web environments: HTML-understanding research uses prompting and fine-tuning to address the special formats and long contexts that make webpages difficult for LLMs to process accurately.RCI prompting demonstrates inherent HTML-understanding ability, while other work improves it through fine-tuning.
- Web environments: Visual-grounding research operates on rendered webpages with multimodal models, and screenshots usually improve performance over text-based LLMs.CogAgent and SeeClick are examples of web-agent frameworks using large multimodal models.
- Personalization resources: Personalized recommendation research is organized into rating prediction, sequential recommendation, explanation generation, review generation, and direct recommendation.The survey compares existing studies and compiles relevant datasets for recommendation and search.
- Personalization resources: Common datasets support recommendation, search, task-oriented dialogue, and user-persona modeling, including MultiWOZ and PersonaChat.PersonaChat contains profiles covering background, preferences, and personality traits for coherent contextual multi-turn dialogue.