Source-linked AI summary
Character-LLM: A Trainable Agent for Role-Playing
Yunfan Shao, Linyang Li, Junqi Dai, Xipeng Qiu
TL;DR
Character-LLM addresses the limits of prompting for simulating specific people with their experiences, characteristics, and emotions. It reconstructs profile-based experiences into scenes, trains character-specific agents, and evaluates them through interviews; the agents memorize characters and experiences, but remain constrained by limited data and hallucinations from broad knowledge.
Problem
Simple prompting is insufficient for simulating a person’s deeper thinking and experience, while broad pretrained knowledge can produce character-inconsistent information.
Method
Character-LLM reconstructs profiles as detailed experiences, uploads those scenes through training, and uses protective experiences to limit unrelated knowledge.
Results
Interview evaluation shows that trained agents can memorize their characters and personal experiences, while protective experiences successfully mitigate character hallucination.
Takeaways & Limitations
Trainable agents are promising for preserving experiences and host personalities in character simulacra and may support applications such as NPCs and online services.
Takeaways & Limitations
The narrated scenes are insufficient to represent a whole person or even one aspect of a real person, and evaluation lacks standard metrics or protocols.
Abstract
from arXiv · showhide
Large language models (LLMs) can be used to serve as agents to simulate human behaviors, given the powerful ability to understand human instructions and provide high-quality generated texts. Such ability stimulates us to wonder whether LLMs can simulate a person in a higher form than simple human behaviors. Therefore, we aim to train an agent with the profile, experience, and emotional states of a specific person instead of using limited prompts to instruct ChatGPT API. In this work, we introduce Character-LLM that teach LLMs to act as specific people such as Beethoven, Queen Cleopatra, Julius Caesar, etc. Our method focuses on editing profiles as experiences of a certain character and training models to be personal simulacra with these experiences. To assess the effectiveness of our approach, we build a test playground that interviews trained agents and evaluates whether the agents \textit{memorize} their characters and experiences. Experimental results show interesting observations that help build future simulacra of humankind.
1 Introduction
Character-LLM trains agents to simulate specific people by learning profiles, reconstructed experiences, and emotions rather than relying on limited prompts. Interviews indicate that the agents memorize character experiences and personalities, while protective experiences mitigate hallucinations.
- Prompting is insufficient for representing deeper personal thinking and experience, motivating trainable agents for more vivid role-playing.
- Character-LLM trains agents as specific-person simulacra that learn from actual experiences, characteristics, and emotions.
- The framework reconstructs personal profiles into detailed experience scenes, uploads them through supervised fine-tuning, and adds protective experiences.Examples include Beethoven, Cleopatra, and Caesar; reconstructed scenes describe events and feelings that agents can learn.
- Interview-based evaluation compares trained agents with instruction-tuned models and uses labeled scenes and LLM evaluators to assess character identification and experience memorization.
- The results show that Character-LLMs successfully memorize training characters and experiences, while protective experiences mitigate hallucinations.Case studies also examine when agents perform well and when they fail to maintain their trained characters.
- The paper contributes Character-LLM, a training framework with Experience Reconstruction, Experience Upload, and Protective Experiences.
2 Related Work
Related work develops agents that simulate human behavior, including game NPCs and interactive social agents, increasingly using prompted or specialized language models. Character-LLM extends this direction toward trainable character simulacra.
- Earlier work introduced agents designed to provide an illusion of life and perform as humans, including systems for game NPCs and cognitive simulation.
- Prompted language models have been used to generate personas, behaviors, and short natural-language descriptions for social-behavior simulation.
- LLMs also support interactive behaviors between users and simulacra, with game NPCs demonstrating interaction abilities with human players.
- LLM specialization includes instruction understanding and alignment methods such as InstructGPT, RLHF, Alpaca, and Vicuna.
3 Approach
Character-LLM specializes separate character agents by reconstructing profile-grounded experiences into scenes and interactions, then training on those experiences. Protective scenes encourage ignorance outside a character’s capabilities to reduce character hallucination.
- 3 Approach: Experience Upload trains characters from reconstructed experiences rather than curated utterance styles or hand-crafted prompting rules.
- 3.1 Experience Reconstruction: Profiles are converted into flashback scenes by instruction-following LLMs, which are expanded into detailed interactions and character thoughts.
- 3.1.1 Profile Collection: The reconstruction pipeline uses profile descriptions spanning a character’s attributes and significant events as the basis for experience generation.
- 3.1.2 Scene and Interaction: Scenes encode temporal, spatial, and participant context, while interactions represent characters’ cognitive processes, utterances, and actions in plain text.
- 3.2 Protective Experience: Character hallucination occurs when broad pretrained knowledge produces information inconsistent with a character’s identity or era.An ancient Roman character answering a Python question illustrates the mismatch.
- 3.2 Protective Experience: Protective scenes train models to express ignorance when questions exceed the character’s inherent capabilities.
- 3.3 Experience Upload: Each character receives a separately fine-tuned agent using only that character’s experiences, which the authors report enhances role-playing accuracy.
- 3 Approach: The method aims to improve reliability and believability through curated profiles, reconstructed scenes, and protective-scene augmentation.
4 Experiments
The experiments evaluate trainable character simulacra through diverse single- and multi-turn interviews, comparing them with prompt-based instruction-following baselines. Character-LLMs are trained separately for each character using reconstructed experience data.
- Evaluation Setup: Character-LLMs are evaluated through interviews designed to assess the quality of simulacra responses.The evaluation focuses on acting ability rather than performance on specified tasks.
- Training Setup: Each Character-LLM is fine-tuned separately on the experience examples corresponding to its character.The training setup initializes from LLaMA 7B and inserts a concise scene description into each example.
- Evaluation Setup: The study includes more than 100 diverse single-turn and multi-turn interviews for each role.Questions are generated from varied topics and manually examined for one character to remove off-topic items.
- Interview Protocols: Single-turn interviews remove previous-question context to probe the models’ inherent memory and knowledge across many questions.This design reduces the influence of earlier questions during evaluation.
- Interview Protocols: Multi-turn interviews test whether models gradually deviate from the intended character during prolonged performance.ChatGPT acts as interviewer, asking harsh questions and follow-ups when responses lack detail.
- Baselines: The baselines are Alpaca 7B, Vicuna 7B, and ChatGPT, with detailed character descriptions supplied through prompts.Alpaca 7B and Vicuna 7B share the LLaMA 7B backbone used for Character-LLMs.
4.4 LLM as Judges
The paper uses GPT-3.5 as a step-by-step judge to assess character acting across five dimensions. Scores are averaged to represent acting believability on a 7-point Likert scale.
- Evaluation Dimensions: The five dimensions are personality, values, memorization, hallucination, and stability.Figure 4 reports these dimensions using a 7-point Likert scale.
- Evaluation Dimensions: Memorization measures recall of character-specific information about associated people, events, and objects.The criterion emphasizes precise and detailed knowledge relevant to the portrayed character.
- Evaluation Dimensions: Values measure whether the model shares the character’s objectives, values, preferences, and biases when evaluating situations.The dimension focuses on the character’s distinctive perspective rather than generic correctness.
- Evaluation Dimensions: Personality measures whether the model reproduces the character’s thinking or speaking style, tone, emotions, and reactions.The criterion covers how the character would respond under different circumstances.
- Evaluation Dimensions: Hallucination assesses whether the model discards knowledge the character would not possess, while stability assesses consistency during prolonged acting.Stability concerns deviations caused by pre-training or alignment under changing incremental inputs.
- Judging Procedure: GPT-3.5 evaluates each interviewee one dimension at a time using five primary acting dimensions.The judging procedure first presents the criterion for the current dimension before scoring performance.
4.5 Main Results
Character-LLMs outperform same-scale baselines across personality, memorization, hallucination, and stability, while producing vivid, experience-specific responses. Protective experiences reduce hallucinations, but limited experience and difficulty reflecting character values remain challenges.
- Character-LLMs outperform Alpaca 7B and Vicuna 7B across personality, memorization, hallucination, and stability.The authors attribute this to learning character experiences and mimicking the person’s style and tone.
- Character-LLMs produce more vivid responses, recall specific past experiences, and reject unnatural questions compared with same-scale baselines.
- Character-LLMs achieve comparable performance to ChatGPT despite using a 7B model.
- Agents struggle to reflect character values, possibly because the models tend to generate shorter responses.
- Memorization Consistency: In parent interviews, trained simulacra provide Beethoven-specific memories and emotions, whereas Alpaca and Vicuna give general responses.ChatGPT can provide correct information but closely reproduces Wikipedia’s wording.
- Protective Scenes: Fewer than 100 protective scenes per character alleviate hallucination without interfering with other portrayal capabilities.
- Protective Scenes: Protective experiences make agents refuse questions outside the character’s expertise, such as writing Python code.
- Hallucinations remain a critical portrayal issue because they reduce believability and may create security risks.The authors state that prompt engineering and vanilla supervised fine-tuning do not completely resolve hallucinations.
5 Conclusion and Future
The paper presents Character-LLM as an experience-upload framework for training agents to simulate specific people. Its evaluation indicates that trained agents memorize characters and personal experiences, supporting applications including NPCs and online services.
- Character-LLM reconstructs narrative scenes from experiences and trains specific models to act as chosen characters.
- Evaluations using interviews and AI-based judging show that trained agents memorize their characters and personal experiences.
- The resulting simulacra could support applications such as NPCs, online services, and social typing.
- Future work aims to give characters specific actions and enable interaction with people or other agents in a sandbox.
Limitations
The paper identifies limitations in evaluation, data coverage, base-model dependence, and potential harms of character simulacra. These constraints motivate improved protocols, richer sources, stronger models, and safeguards.
- Evaluation Protocols: Character-simulacra evaluation lacks standard metrics and protocols, while assessing personality requires proficient character understanding.The study therefore uses LLM evaluators and massive case analyses, but calls for better evaluation protocols.
- Limited data: Profile-based narrated scenes cannot represent a person’s whole life or even one aspect of a real person.Future work could incorporate biographies, interviews, historical comments, and crafted scenes.
- Base model: Supervised fine-tuning outcomes depend strongly on base-model pretraining distribution, architecture, and scale.The paper suggests exploring trainable agents based on more powerful and larger LLMs.
- Potential Harm: Character simulacra may generate offensive content or enable harmful manipulation when portraying flawed or vicious characters.The paper frames vividness and avoiding negative-thought characters as a trade-off that becomes more important as LLMs strengthen.
Ethics Statement
The paper addresses privacy and ethical risks by using public historical or imaginary characters and controlled, open-source training data. It nevertheless warns that third-party poisoned or negative data could produce harmful agents, requiring strict oversight.
- Data and privacy: The study selects historical figures and imaginary characters from publicly available Wikipedia profiles to avoid privacy and personal-data issues.Experience data are generated from Wikipedia-provided facts using ChatGPT.
- Risk and oversight: Third parties could inject poisoned or negative training data, causing trained agents to produce negative effects.The paper recommends strict censoring and supervision to increase potential benefits relative to negative influence.
A Prompts Demonstration
The appendix documents prompts used throughout Character-LLM’s reconstruction, agent-role-playing, interviewing, and evaluation procedures. It lists separate judge prompts for five evaluation dimensions.
- Experience Reconstruction: Experience Reconstruction prompts specify scene extraction and experience completion, while another prompt generates protective experience.These prompts are presented in Tables 4 and 5.
- Agent and interviewer prompts: Meta prompts define behavior for trainable agents, baseline models, and ChatGPT interviewers in multi-turn interviews.The paper presents these prompts in Table 6.
- Evaluation prompts: ChatGPT Judge prompts separately evaluate memorization, personality, values, hallucination, and stability.The five dimension-specific prompts appear in Tables 7 through 11.
B Qualitative Generation Examples
The appendix provides qualitative examples of Character-LLM responses in both single-turn and multi-turn interactions. These examples span inductive prompts and conversations on different topics.
- Single-Turn Interactions: Tables 12–19 show single-turn responses from different agents to inductive prompts.The examples illustrate isolated responses across agents and prompts.
- Multi-Turn Interactions: Tables 20–25 show multi-turn interaction examples involving the agents across different topics.These examples document conversational behavior over multiple exchanges.
C Selection of Training Examples
The training examples include prompts for reconstruction, protective-experience generation, evaluation, and interviews across multiple simulated characters. The examples also contain multi-turn dialogue exploring character experiences, opinions, and historical or fictional events.
- Prompts cover experience reconstruction and protective-experience generation for constructing training examples.
- Evaluation prompts assess memorization, personality, values, hallucination, and stability in character simulacra.
- Single-turn interview examples compare methods simulating Julius Caesar, Spartacus, Lord Voldemort, Isaac Newton, Socrates, Martin Luther King Jr., Cleopatra VII, and Hermione Granger.
- Additional single-turn examples cover Cleopatra VII and Hermione Granger, while multi-turn examples feature Julius Caesar, Ludwig van Beethoven, and Martin Luther King Jr.
- The Hermione and Voldemort dialogues probe difficult experiences, views of Voldemort, his rise to power, and lessons for preventing similar events.
- The dialogues include Hermione’s assessments of Voldemort’s influence, motivations, key events, and possible preventive actions.