Source-linked AI summary
RoleLLM: Benchmarking, Eliciting, and Enhancing Role-Playing Abilities of Large Language Models
Zekun Moore Wang, Zhongyuan Peng, Haoran Que, Jiaheng Liu, Wangchunshu Zhou, Yuhan Wu, Hongcheng Guo, Ruitong Gan, Zehao Ni, Jian Yang, Man Zhang, Zhaoxiang Zhang, Wanli Ouyang, Ke Xu, Stephen W. Huang, Jie Fu, Junran Peng
TL;DR
Role-playing LLMs face limited open-source optimization, coarse role granularity, scarce benchmarks, and closed-model cost and context constraints. RoleLLM addresses these issues with role profiles, Context-Instruct, RoleGPT, RoleBench, and RoCIT, producing tuned open-source models whose role-playing abilities are comparable with RoleGPT in some cases. The framework is limited to single-turn question answering, English and Chinese, and a selection of public scripts.
Problem
Open-source LLMs lack role-playing optimization, while existing methods have limited character-level granularity, insufficient data and benchmarks, and API and context costs.
Method
RoleLLM constructs role profiles, extracts role-specific knowledge with Context-Instruct, imitates speaking styles with RoleGPT, and applies RoCIT to open-source models.
Results
RoleLLaMA and RoleGLM show strong role-playing abilities and are comparable with RoleGPT using GPT-4 in some cases.
Takeaways & Limitations
RoleBench provides a fine-grained benchmark and open-source instruction-tuning data for character-level role-playing.
Takeaways & Limitations
The framework is designed for single-turn question answering and is limited to English and Chinese and a selection of publicly available scripts.
Abstract
from arXiv · showhide
The advent of Large Language Models (LLMs) has paved the way for complex tasks such as role-playing, which enhances user interactions by enabling models to imitate various characters. However, the closed-source nature of state-of-the-art LLMs and their general-purpose training limit role-playing optimization. In this paper, we introduce RoleLLM, a framework to benchmark, elicit, and enhance role-playing abilities in LLMs. RoleLLM comprises four stages: (1) Role Profile Construction for 100 roles; (2) Context-Based Instruction Generation (Context-Instruct) for role-specific knowledge extraction; (3) Role Prompting using GPT (RoleGPT) for speaking style imitation; and (4) Role-Conditioned Instruction Tuning (RoCIT) for fine-tuning open-source models along with role customization. By Context-Instruct and RoleGPT, we create RoleBench, the first systematic and fine-grained character-level benchmark dataset for role-playing with 168,093 samples. Moreover, RoCIT on RoleBench yields RoleLLaMA (English) and RoleGLM (Chinese), significantly enhancing role-playing abilities and even achieving comparable results with RoleGPT (using GPT-4).
1 Introduction
Role-playing enables LLMs to simulate characters with distinct attributes and conversational styles, but existing models and methods face limitations in optimization, granularity, data, benchmarking, and cost.
- Role-playing enables LLMs to simulate characters with distinct attributes and conversational styles for more nuanced interactions.
- Existing open-source LLMs are mainly trained on general domains and lack specific optimization for role-playing.
- Closed-source SOTA models constrain role-playing through high API costs, unavailable fine-tuning, and limited context windows.
- Prior methods mainly target coarse-grained traits, professions, or personas rather than fine-grained character-level role-playing.
- The field lacks high-quality, diverse, extensive open-source datasets and benchmarks for evaluating role-playing.
🔮RoCIT
RoleLLM combines profile construction, knowledge extraction, speaking-style imitation, and role-conditioned tuning to benchmark and enhance role-playing in open-source models.
- RoleLLM uses four stages: role profile construction, Context-Instruct, RoleGPT, and RoCIT.Context-Instruct extracts role-specific knowledge, RoleGPT imitates speaking styles, and RoCIT tunes open-source models.
- RoleBench contains 168,093 role-playing samples generated by Context-Instruct and RoleGPT for fine-tuning open-source models.
- RoleLLaMA and RoleGLM are obtained by role-conditioned instruction tuning of LLaMA and ChatGLM2 on English and Chinese RoleBench data.
- The evaluation uses Rouge-L, GPT, and human assessments for speaking-style imitation, answering accuracy, and role-specific knowledge capture.
- RoleBench markedly improves models’ role-playing abilities, with competitive results relative to RoleGPT in some cases.
2 Methods
The methods combine dialogue engineering, role-specific knowledge extraction, and role-conditioned tuning to improve character style and knowledge while supporting role customization.
- RoleLLM’s methods comprise RoleGPT, Context-Instruct, and RoCIT with system-instruction-based role customization.
- RoleGPT: RoleGPT uses role descriptions, catchphrases, task instructions, and retrieved dialogue pairs to imitate characters’ speaking styles.The approach targets lexical consistency and dialogic fidelity, while retrieval augmentation can be limited by sparse and noisy profiles.
- Context-Instruct: Context-Instruct segments role profiles, generates question-confidence-answer candidates, and filters or deduplicates low-quality data.Confidence scores help reduce incomplete or hallucinated script-based questions.
- RoCIT: RoCIT fine-tunes LLaMA and ChatGLM2 on general and role-specific augmented data, embedding role-specific knowledge into model weights.
- RoCIT: Context-Instruct data has less distinct speaking style than RoleGPT data because it uses fewer demonstrations for dialogic fidelity.
- RoCIT: System instructions customize roles using names, descriptions, catchphrases, and role-playing task instructions during inference.
3 RoleBench
RoleBench is constructed from 100 character profiles and processed into general and role-specific instruction data, with quality and diversity analyses supporting its benchmark role.
- Data Construction: RoleBench construction includes role selection, profile construction, general-instruction sampling, raw-data generation, and dataset cleaning.
- Data Construction: The dataset selects 100 distinctive characters from diverse English and Chinese scripts and builds profiles with descriptions, catchphrases, and structured dialogues.
- Data Construction: RoleBench includes English and Chinese general and role-specific subsets generated using GPT-4 and Context-Instruct.
- Quality: Expert review of randomly sampled general and role-specific instances found that most samples were high quality.
- Data Analysis: RoleBench diversity is analyzed through instruction verb-noun structures, role classes, word clouds, and instruction-response length distributions.
- Examples: An example pairs Dr. Hannibal Lecter with a list-length question and a response written in a character-specific style.
4 Experiments
Experiments evaluate role-playing across instruction and role generalization, using Rouge-L, GPT-based, and human assessments. Ablations examine knowledge injection, prompting, customization, and model scaling.
- Experiment Setup: Rouge-L, GPT-based evaluation, and human evaluation assess response accuracy, speaking-style imitation, and role-specific knowledge.RAW measures instruction accuracy, CUS measures role-associated speaking style, and SPE measures role-specific knowledge.
- Main Experiments: RoleLLaMA improves English instruction-generalization performance over instruction-tuned, base, and earlier role-playing models across CUS, RAW, and SPE.Direct script-data tuning can underperform the base model, while general-purpose instruction tuning helps but does not fully close the role-playing gap.
- Main Experiments: RoleGLM fine-tuning with RoleBench further enhances Chinese role-playing across all metrics and slightly surpasses RoleGPT.ChatGLM2 already performs notably on role-specific knowledge before enhancement.
- Role Generalization: On 10 unseen English roles, RoleLLaMA improves speaking-style imitation and maintains response accuracy, but shows no noticeable SPE improvement over baselines.The authors attribute the SPE limitation to models lacking prior knowledge of unseen roles.
- Ablation Study: Context-Instruct substantially increases role-specific knowledge, whereas retrieval augmentation may distract models and reduce robustness because retrieval sources are noisy.The comparison is conducted for instruction generalization and role-specific knowledge injection.
- Ablation Study: Few-shot dialogue engineering outperforms few-shot prompt engineering, and both outperform zero-shot prompting in GPT-4 evaluation.Win Rate counts how often a method ranks first, while Avg. Ranking measures its mean rank among the three strategies.
- Scaling Law: Larger RoleLLaMA models produce better role-playing results across the tested 7B, 13B, and 33B sizes.The scaling analysis is presented as the role-playing scaling law.
5 Related Work
Recent work uses role-playing to make LLMs more vivid, interactive, personalized, and capable of addressing complex tasks.
- Role-playing research explores LLM customization for more vivid and interactive behavior.
- Role-playing also supports personalized interactions and complex-task assistance.
6 Conclusion
RoleLLM provides a framework to benchmark, elicit, and enhance role-playing in LLMs. It constructs role resources and training methods, producing models with strong abilities comparable to RoleGPT using GPT-4.
- RoleLLM benchmarks, elicits, and enhances role-playing abilities in LLMs.
- The framework constructs 100 role profiles, uses RoleGPT for speaking-style imitation, and applies Context-Instruct for role-specific knowledge extraction.
- RoleLLaMA and RoleGLM are obtained through role-conditioned instruction tuning on RoleBench.
- RoleLLaMA and RoleGLM show strong role-playing abilities and are comparable with RoleGPT using GPT-4.
Limitations
The framework is designed for single-turn question-answering and is therefore limited in multi-turn dialogue scenarios; its resources currently cover English and Chinese and selected public scripts.
- The framework targets single-turn question-answering, limiting applicability to multi-turn dialogue scenarios.
- The research and resource releases are currently confined to English and Chinese.
- The dataset is limited to selected publicly available scripts, with planned expansion to more scripts and characters.
Ethics Statement
Role-playing systems may produce sensitive or harmful content when misused, including content reflecting violent, explicit, or discriminatory character traits. The paper recommends moderation, bias mitigation, transparency, and user feedback mechanisms for responsible use.
- Role-playing may lead to jailbreaking, and misuse could generate sensitive or harmful content.
- Generated content may reflect violent, explicit, or discriminatory traits of certain characters.
- The paper recommends advanced moderation tools to detect and filter sensitive, harmful, or inappropriate outputs.
- Bias detection and mitigation are recommended to reduce stereotypes and discrimination in outputs.
- Transparency about RoleLLM’s design, capabilities, limitations, and decision-making processes is recommended.
- A feedback system is recommended so users can report ethical concerns and problematic outputs, supporting improvement and accountability.
A.2 Comparative Case Study
The case studies compare models on factual answering and character-specific speaking style. RoleLLaMA and RoleGPT generally preserve stronger role traits, while Context-Instruct adds role-related knowledge to RoleLLaMA.
- Style and factual answering: RoleGPT and RoleLLaMA correctly answered the weight-ordering question while Character.AI and Alpaca answered incorrectly.Both stronger models also carried a strong character style; RoleLLaMA additionally incorporated character-related knowledge content.
- Style and factual answering: RoleLLaMA showcased Twilight Sparkle’s lively and passionate style more clearly than the other models in the educational-requirements example.The example’s response included the experience requirement while retaining the character’s distinctive style.
- Style and factual answering: RoleLLaMA and RoleGPT maintained strong character styles while answering the Jack Torrance ingredient question, unlike the other models.The analysis specifically characterizes their responses as rude and impolite, whereas the other models lacked distinct character-style features.
- Cross-language examples: RoleGPT and RoleGLM displayed Imperial Concubine Hua’s arrogant and overbearing characteristics more clearly than the other models.Their Chinese responses were less plain and better reflected the described classical character.
- Role-specific knowledge: RoleLLaMA and Character.AI identified Captain Jack’s Black Pearl, while RoleGPT and Alpaca selected different possessions.The analysis attributes RoleLLaMA’s emotionally richer response and its knowledge improvement to Context-Instruct.
C More Details on RoleBench Construction
RoleBench is constructed from curated role profiles, sampled general instructions, generated role-conditioned responses, and systematic cleaning. Its design supports fine-grained character-level role-playing data across English and Chinese roles.
- Role selection and profiling: The authors select English and Chinese scripts, then identify representative characters and audit their speaking-style distinctiveness.The source collection includes 661 English scripts from NLP Movie Scripts, 255 from SummScreen, and 24 manually curated Chinese scripts.
- Role selection and profiling: Each role profile combines an author-verified description, catchphrases, and structured dialogue parsed from the character’s scripts.The dialogue is organized hierarchically from individual turns to dialogue rounds and acts or episodes.
- Instruction generation: The construction samples 1,500 English and 1,479 Chinese general instructions, removing duplicates using BM25 similarity.English instructions come from Super-NaturalInstruct, UltraChat, and Alpaca; Chinese instructions come from COIG and BELLE.
- Instruction generation: RoleGPT generates style- and knowledge-conditioned responses for general instructions, while Context-Instruct generates role-specific instructions and responses together.The resulting RoleBench data includes general and role-specific subsets that are subsequently cleaned.
- Data cleaning: RoleBench cleaning removes responses with identity disclosure, role-name disclosure, rejection, or incomplete sentence endings.For RoleBench-general, only the five ground-truth candidates are cleaned.
- Dataset statistics: The dataset documentation includes role categories, instruction verb-noun structure, length distributions, and detailed basic statistics.These properties are presented through Figures 3–5 and Table 9.
F More Details on Evaluation Protocol
The evaluation protocol measures instruction and role generalization using separate train-test splits, automated metrics, GPT comparisons, and human evaluation. Character.AI receives a smaller manually collected evaluation subset because it lacks API access.
- Dataset splits: Instruction generalization uses a 4:1 train-test ratio with 1,200 training and 300 test instructions in RoleBench-general.Role-specific test data includes up to 50 samples per role.
- Dataset splits: Role generalization uses English role-based splits with an 8:1 ratio of 80 training roles to 10 test roles.The protocol separately evaluates unseen instructions and unseen roles.
- Metrics and evaluators: Evaluation uses Rouge-L, GPT, and human scores to assess model responses.The stated evaluation dimensions are speaking-style imitation, answering accuracy, and role-specific knowledge capture.
- Metrics and evaluators: Character.AI is evaluated on a reduced manually operated subset because it does not provide API access.The same subset is also used for human evaluation by three NLP graduate students.
G Further Experimental Analysis
Further analyses examine data mixing, Context-Instruct, cross-cultural training balance, and prompting formats. The results support mixed training, show the importance of balanced cultural data, and illustrate how prompt demonstrations are organized.
- Data mixing: Mixed training on RoleBench-general and RoleBench-specific produces the best-balanced results among the tested data-mixing strategies.RoleBench-general is generated by RoleGPT, whereas RoleBench-specific is produced through Context-Instruct.
- Role generalization: Context-Instruct achieves performance comparable to instruction generalization in the role-generalization setting.The comparison is reported alongside Retrieval Augmentation in Table 11.
- Cross-lingual and cross-cultural training: Balanced training with 5 Chinese and 5 English characters nearly matches single-language models across linguistic contexts.Training with 5 Chinese and 95 English characters reaches English saturation but performs much worse in Chinese.
- Cross-lingual and cross-cultural training: The cross-lingual results indicate that training-data balance across language cultures is important for preserving China-specific cultural features.The authors attribute the Chinese-performance gap under English-heavy training to the reduced proportion of ancient-Chinese expressions.
- Prompting formats: RoleGPT prompt formats combine role descriptions and catchphrases with either no demonstrations, concatenated demonstrations, or explicit user-assistant dialogue turns.The templates are shown for English and Chinese zero-shot, few-shot, and dialogue-engineering variants.
- Context-Instruct analysis: Context-Instruct evaluates generated question-context-answer candidates for factualness and completeness before using them in role-specific data construction.The examples classify questions and provide rationales for the ratings.