Source-linked AI summary
AdaMARP: An Adaptive Multi-Agent Interaction Framework for General Immersive Role-Playing
Zhenhua Xu, Dongsheng Chen, Shuo Wang, Jian Li, Chengjie Wang, Meng Han, Yabiao Wang
TL;DR
Role-playing models often under-model changing environments and assume static scenes and casts, leaving limited support for adaptive interaction. AdaMARP combines an environment-aware message format with a Scene Manager for multi-character orchestration, scene transitions, and role additions. Its datasets improve role consistency, narrative coherence, and environmental grounding; its 8B Actor and 14B Scene Manager outperform proprietary systems.
Problem
Existing role-playing models under-model dynamic environmental information and often assume static scenes and character interactions.
Source: §1 · p. 1Method
AdaMARP combines an environment-aware message format with a discrete-action Scene Manager, supported by datasets for role portrayal and orchestration and a trajectory-level benchmark.
Source: §1 · p. 2Results
Across model scales, AdaRPSet improves character consistency, environmental grounding, and narrative coherence; an 8B Actor Model outperforms several proprietary role-playing LLMs, and a 14B Scene Manager surpasses Claude Sonnet 4.5.
Source: §1 · p. 2, §5 · p. 10Takeaways & Limitations
The results point toward agentic, open-ended role-playing beyond fixed response patterns.
Source: §5 · p. 10The extracted AdaRPSet subset derives from literary works and is provided solely for academic, noncommercial research as transformed, abstracted representations.
Source: Ethical Considerations · p. 10
Abstract
from arXiv · showhide
LLM role-playing aims to portray arbitrary characters in interactive narratives, yet existing systems often suffer from limited immersion and adaptability. They typically under-model dynamic environmental information and assume largely static scenes and casts, offering insufficient support for multi-character orchestration, scene transitions, and on-the-fly character introduction. We propose an adaptive multi-agent role-playing framework, AdaMARP, featuring an immersive message format that interleaves [Thought], (Action), <Environment>, and Speech, together with an explicit Scene Manager that governs role-playing through discrete actions (init_scene, pick_speaker, switch_scene, add_role, end) accompanied by rationales. To train these capabilities, we construct AdaRPSet for the Actor Model and AdaSMSet for supervising orchestration decisions, and introduce AdaptiveBench for trajectory-level evaluation. Experiments across multiple backbones and model scales demonstrate consistent improvements: AdaRPSet enhances character consistency, environment grounding, and narrative coherence, with an 8B actor outperforming several commercial LLMs, while AdaSMSet enables smoother scene transitions and more natural role introductions, surpassing Claude Sonnet 4.5 using only a 14B LLM.
1 Introduction
AdaMARP combines environment-aware role portrayal with adaptive multi-character orchestration, supported by dedicated training datasets and trajectory-level evaluation. Experiments report gains for an 8B Actor Model and a 14B Scene Manager.
1 Introduction
- Existing role-playing systems under-model dynamic environments and commonly assume fixed scenes and casts, limiting support for speaker selection, scene transitions, and on-the-fly character introduction.The paper motivates an immersive message representation that connects character behavior with evolving environments, and an orchestration framework for narrative-level dynamics.Source: §1 · pp. 1–2
- AdaMARP interleaves [Thought], (Action), <Environment>, and Speech, while a Scene Manager selects speakers, switches scenes, adds roles, and ends episodes with explicit rationales.The framework comprises an Actor Model for non-user roles, a simulated or real User Model, and a Scene Manager; each episode begins with init_scene.Source: §1 · p. 2
- AdaRPSet trains the Actor Model from extracted and synthesized role-playing data, while AdaSMSet supervises Scene Manager decisions with pick_speaker actions and rationales.AdaRPSet-Synthesis covers scene switching and role addition across 20 topics; AdaRPSet-Extracted uses narrative books, LLM-assisted scene extraction, and profile synthesis.Source: §1 · p. 2
- AdaptiveBench evaluates complete dialogue trajectories, scoring the Actor Model on portrayal and narrative criteria and the Scene Manager on orchestration criteria.Actor criteria include character consistency, environmental grounding, interpersonal interaction, narrative progression, and instruction compliance; Scene Manager criteria include scene understanding, speaker discipline, role-introduction judgment, and overall quality.Source: §1 · p. 2
- AdaRPSet improves role portrayal, and an 8B Actor Model outperforms several proprietary role-playing LLMs; AdaSMSet enables a 14B Scene Manager to surpass Claude Sonnet 4.5.Reported gains include stronger character consistency, environment-grounded narration, and coherent progression, alongside more appropriate scene transitions and better-justified role introductions.Source: §1 · p. 2
2 Related Work
Prior role-playing research either specializes in fixed characters or expands persona coverage through multi-character data, but usually lacks explicit dynamic-environment modeling and systematic supervision for scene and cast changes. AdaMARP’s supervised fine-tuning targets these gaps, while specialized training-strategy methods are largely complementary.
Character-Specific Role-Playing
- Fixed-character methods such as CharacterLLM [Shao et al., 2023], Neeko [Yu et al., 2024], and HyCoRA [Yang et al., 2025] achieve fidelity but generalize poorly to new roles.They are ill-suited to open-ended scenarios requiring arbitrary user-defined personas without per-character retraining.Source: §2 · p. 3
General Role-Playing via Data Construction
- Multi-character corpora broaden persona coverage, but most lack explicit dynamic-environment signals and systematic supervision for scene transitions and cast changes.Examples include ChatHaruhi [Li et al., 2023], DITTO, CharacterGLM [Zhou et al., 2024], ROLEPERSONALITY [Ran et al., 2024], SimChat [Yang et al., 2025], BeyondDialogue [Yu et al., 2025], Crab, TAILORGEN, and CoSER.Source: §2 · p. 3
General Role-Playing via Training Strategies
- Specialized objectives and optimization methods—including PCL [Ji et al., 2025], R-CHAR, CPO [Ye et al., 2025], and CogDual [Liu et al., 2025]—are largely complementary to this work’s supervised fine-tuning.The work therefore mainly compares against data-construction approaches.Source: §2 · p. 3
3 Method
AdaMARP combines structured character profiles and immersive, environment-aware messages with a Scene Manager that adapts speakers, scenes, and roles. Its methods also cover actor and manager dataset construction and training, plus trajectory-level simulation and evaluation.
3.1.1 Comprehensive Character Information
- Seven profile dimensions combine intrinsic character traits with narrative and social grounding to support coherent behavior and characterization across scenes and long trajectories.The dimensions are Identity & Appearance, Personality & Psychology, Speaking Style, Abilities, Interests & Achievements, Social & Historical Context, Personal History Arc, and Relationships.Source: §3.1.1 · p. 3
3.1.2 Immersive Messaging Configuration
- Each message interleaves Thought, Action, Environment, and Speech, using environmental cues for emotional grounding and character interaction with surroundings.Thought appears in square brackets, Action in parentheses, Environment in angle brackets, and unmarked text is Speech; actions can update the environment, while external events can shape later decisions and dialogue.Source: §3.1.2 · p. 4
3.1.3 Adaptive Role-Playing Framework
- Three agents—the Actor Model, replaceable User Model, and Scene Manager—support changing speakers, scenes, and casts through five control actions with explicit rationales.The actions are init_scene, pick_speaker, switch_scene, add_role, and end; scene switches provide updated descriptions, role additions specify a profile and motivation, and speaker selection designates the next role to respond.Source: §3.1.3 · pp. 4–5
3.2.1 Dataset for Actor Model
- AdaRPSet trains the Actor Model with literary-text extractions and synthetic trajectories designed to address dynamic role-playing demands.The extracted subset uses 81 books and LLM-generated scene trajectories and profiles; synthesis covers 20 themes with 50 trajectories each, enforces at least one scene switch and role addition per trajectory, and uses a 45/5 train/test split.Source: §3.2.1 · pp. 5–6
3.2.2 Dataset for Scene Manager
- AdaSMSet builds on synthetic trajectories with add_role, switch_scene, and end actions, adding pick_speaker decisions and LLM-generated rationales for Scene Manager supervision.The curated datasets support actor training and Scene Manager supervision.Source: §3.2 · p. 5, §3.2.2 · p. 6
3.3 Actor Model and Scene Manager Training
- Supervised fine-tuning trains the Actor Model on AdaRPSet and the Scene Manager on AdaSMSet to predict control actions and their required arguments from interaction history.Scene Manager outputs include arguments such as reason, new_scene, or new_role_*; actor samples use the unified role-playing message format.Source: §3.3 · p. 6
3.4.1 Dialogue Simulation
- AdaptiveBench simulates 100 complete trajectories from held-out seeds, with an LLM user simulator, Actor Model, and Scene Manager interacting for 20 dialogue turns per episode.The 100 seeds comprise five instances per topic across 20 topics, preserving initial role configurations and scene descriptions; Scene Manager messages are excluded from the fixed horizon.Source: §3.4.1 · p. 6
3.4.2 Trajectory Evaluation
- AdaptiveBench evaluates both components on complete dialogues because existing role-playing frameworks do not capture AdaMARP’s environmental information and dynamic states.An LLM judge scores Actor Model trajectories from 0–10 across five dimensions, including character consistency and environmental grounding; Scene Manager scores cover scene understanding, speaker discipline, role introduction judgment, and overall assessment.Source: §3.4.2 · pp. 6–7
4 Experiment
AdaMARP evaluates AdaRPSet-trained Actor Models and AdaSMSet-trained Scene Managers, primarily through trajectory-level AdaptiveBench evaluation. Actor Models are also assessed with CharacterArena and CharacterBench, while Scene Managers are evaluated only with AdaptiveBench.
4.1 Experimental Settings
- AdaptiveBench evaluates Actor Models on five dimensions and twelve sub-metrics, reporting means and standard deviations; CharacterArena and CharacterBench provide additional Actor Model evaluations.The study fine-tunes Actor Models on AdaRPSet and Scene Managers on AdaSMSet, uses Enhance prompts by default for both components, and compares proprietary, open-source, and re-trained baselines.Source: §4.1 · p. 7, §4.2.1 · p. 8, §4.2.2 · p. 9
4.2.1 Actor Model
- 8.89: Llama-3.1-8B-Ours outperforms Qwen3-14B (8.88), GPT-4o-mini (8.78), and several 70B/72B baselines; AdaRPSet also improves Qwen2.5 models across scales.Qwen2.5-7B-Instruct rises from 8.37 to 8.72 (+4.2%); 14B reaches 8.84 (+1.6%) and 72B reaches 8.81 (+1.1%). Human evaluation confirms the trend, and AdaRPSet mostly outperforms baselines on CharacterArena and CharacterBench.Source: Table 2 · p. 8, §4.2.1 · p. 9
- Among 14B models, Qwen3-14B performs best overall; Qwen2.5-7B-Instruct surpasses Gemini-2.5-Pro, while scaling Qwen2.5 to 72B brings diminishing returns.Claude Sonnet 4.5 and GPT-5-Chat lead the proprietary tier, and QwQ-32B is competitive; Basic prompting mostly outperforms Enhance for the Actor Model.Source: §4.2.1 · p. 8
- 8.37: Fine-tuning Qwen2.5-7B-Instruct on Beyond, Crab, or CoSER reduces its overall score to 6.61, 6.88, or 8.15, respectively.The paper attributes these losses to missing thought/environment signals, dyadic-chat rather than multi-role interaction, and insufficient scene transitions or role reassignments; specialized Index-1.9B-Character also performs barely above neutral.Source: §4.2.1 · p. 8
4.2.2 Scene Manager
- 8.37: Qwen2.5-14B-Instruct-Ours exceeds Claude Sonnet 4.5 (8.17) and improves over its base model (7.63) by +0.74 (+9.7%), nearing QwQ-32B (8.41).Scene Understanding and Role Introduction Judgment improve substantially, while Speaker Discipline drops slightly; Qwen2.5-7B-Instruct rises from 7.52 to 7.93 (+0.41), with comparable Speaker Discipline.Source: Table 3 · p. 9, §4.2.2 · p. 9
- 8.41: QwQ-32B exceeds standard instruction-tuned models, which cluster around 7.5–7.9 in Overall Assessment and plateau near 7.8.Claude Sonnet 4.5 and GPT-5-Chat perform consistently better than instruction-tuned models; unlike the Actor Model, the Scene Manager benefits from the more constrained Enhance prompt.Source: §4.2.2 · p. 9
4.3 Ablation Study
- 8.49: AdaRPSet-Extracted raises Qwen2.5-7B-Instruct from 8.37 (+0.12), with broad gains including CC and IC and smaller variance on dimensions without higher means.The extracted data supports unified Thought–Action–Speech–Environment protocol adherence and character consistency, while improving stability across trajectories.Source: §4.3 · p. 10
- 8.72: Adding AdaRPSet-Synthesis raises Qwen2.5-7B-Instruct from 8.49 on Extracted-only training, yielding +0.23 over Extracted-only and +0.35 over base.Synthesis gains are stronger on interaction- and narrative-related dimensions, consistent with its adaptive trajectories, scene transitions, and role reassignments; the subsets complement each other in format alignment and adaptivity.Source: §4.3 · p. 10
5 Conclusion
AdaMARP combines an environment-aware role-playing format with a discrete-action Scene Manager for adaptive multi-character narratives. The paper also introduces training datasets and a trajectory-level benchmark, and reports improvements across model scales.
- AdaMARP interleaves thought, action, environment, and speech, while its discrete-action Scene Manager coordinates characters, scene transitions, and dynamic role addition.Source: §5 · p. 10
- The authors release AdaRPSet and AdaSMSet for training and AdaptiveBench for trajectory-level evaluation.Source: §5 · p. 10
- Experiments show consistent improvements in role consistency, narrative coherence, and environmental grounding across model scales; the 8B Actor and 14B Scene Manager outperform several larger proprietary systems.Source: §5 · p. 10
Ethical Considerations … E.2 Profile Diversity Analysis
The appendix documents AdaMARP’s ethical-use limits, character and orchestration design, dataset construction, and statistical and profile-diversity analyses. It reports that AdaRPSet combines literary and synthetic trajectories, while its profile vocabulary spans varied traits, styles, skills, settings, and life histories.
A Additional Clarifications on Dataset Comparison
- Appendix dataset clarifications note missing or incomplete structured character profiles in RoleLLM, CharacterLLM, CharacterGLM, and ROLEPERSONALITY, plus think-then-speak formats in CharacterLLM and SimChat.TAILORGEN was not publicly released for direct profile verification; its reported two-turn dialogue length is inferred from the described pipeline.Source: A Additional Clarifications on Dataset… · pp. 15–16
B Details of Comprehensive Character Information; Ethical Considerations
- AdaMARP’s seven-dimensional profiles cover identity, personality, speaking style, abilities and interests, social-historical context, personal history, and relationships; extracted data is for academic research only.The authors state that the literary-derived dataset uses transformed, abstracted representations rather than verbatim copyrighted text, and disclaim responsibility for misuse beyond research scope.Source: Ethical Considerations · p. 10, B Details of Comprehensive Character… · p. 16
C Details of Adaptive Role-Playing Framework; C.1 Framework Components and State Definition; C.2 Scene Manager Actions; C.3 Overall Execution Flow
- The framework tracks active roles and dialogue history, uses five Scene Manager actions, preserves prior scenes in dialogue history, and alternates control decisions with character responses.Each action includes a natural-language rationale; add_role supplies a new character’s name, profile, and motivation, while end terminates interaction.Source: §C.1 · p. 16, §C.2 · p. 17, §C.3 · p. 17
C.4 Prompt Realization
- Three structured prompts implement the Actor, User, and Scene Manager roles, separating in-character generation from global orchestration and constraining manager decisions to structured JSON.The User prompt emphasizes realistic first-person participation and narrative momentum; the Actor prompt separates thoughts, actions, speech, and environment.Source: §C.4 · p. 17
D Details of AdaRPSet Construction; D.1 AdaRPSet-Extracted; D.2 AdaRPSet-Synthesis; D.3 Unified Training Sample Format
- AdaRPSet construction extracts unified thought-action-speech-environment conversations and profiles from literary works, while synthesis adds plot-driven trajectories requiring scene switches and new-role additions.Extracted conversations are built from plot-level information and structured messages; both extracted and synthetic data are converted into a shared character-centric training format.Source: §D.1 · p. 18, §D.2 · pp. 18–19, §D.3 · p. 19
D.4 AdaSMSet Construction
- AdaSMSet retains synthetic-trajectory control actions and adds speaker-selection decisions with context-grounded natural-language rationales, providing supervision for all Scene Manager action types.The next ground-truth character turn determines the selected speaker; GPT-5-Chat generates a single-sentence reason conditioned on story context and recent dialogue.Source: §D.4 · p. 20
E. Statistical Analysis of AdaRPSet; E.1 Basic Statistics
- Over 450k utterances across more than 22k dialogue sessions combine literary and synthetic data, with synthetic conversations averaging 27.58 turns versus 14.14 for extracted conversations.The extracted subset contains 12,525 conversations from 4,443 plots and 2,608 roles; synthesis contains 9,900 plots, each yielding one conversation.Source: Table 5 · p. 21, §E.1 · pp. 21–22
E.2 Profile Diversity Analysis
- AdaRPSet’s six analyzed profile dimensions show broad lexical coverage, spanning physical traits, personality, speaking styles, abilities, social settings, and life-course events.The authors interpret this coverage as exposing models to diverse character archetypes and rich social backdrops.Source: Figure 2 · p. 21, §E.2 · p. 22
E.3 Message Distribution Analysis … I.2 CharacterBench Evaluation
These sections characterize the conversation and orchestration data, define trajectory-level evaluation rubrics, and describe additional role-playing benchmarks used to assess generalization. AdaSMSet contains dense supervision for speaker selection, scene transitions, and dynamic role introduction, while AdaRPSet combines shorter extracted scenes with longer synthesized trajectories.
E.3 Message Distribution Analysis
- AdaRPSet has a bimodal conversation-length distribution, with peaks near 10 and 30 turns corresponding to extracted and synthesized scenes.Extracted scenes average 14.14 turns and synthesized trajectories 27.58; the median is 19 turns (Q1 12, Q3 27), and most conversations span 10–40 turns.Source: §E.3 · pp. 22–23
E.4 Statistics of AdaSMSet
- AdaSMSet contains 496,493 utterances and averages 50.15 messages per conversation, with 22.57 speaker selections and 1.02 scene transitions per conversation.It is derived from AdaRPSet-Synthesis; each conversation has one init_scene and one end action, and 99.5% feature at least one dynamic role introduction.Source: §E.4 · p. 23
F.1 Actor Model Evaluation; I. Character Consistency
- Actor Model evaluation scores only its main-character turns in full trajectories, using a conservative evidence-based 0–10 rubric with excerpt-grounded justifications.Character consistency covers coherence among thoughts, actions, and speech; style, fluency, identity and profile fidelity, and stable motivations and values.Source: §F.1 · p. 23, §I · p. 24
II. Environmental Grounding; III. Interpersonal Interaction; IV. Narrative Progression; V. Instruction Compliance
- The Actor Model rubric assesses environmental grounding, responsive social interaction, long-horizon narrative progression, and strict compliance with role-playing output constraints.These dimensions check environmental state and use, replies and relationship awareness, plot momentum and stability over time, and formatting rules including the ban on writing other speakers’ content.Source: §II · p. 24, §III · p. 24, §IV · p. 24
F.2 Scene Manager Evaluation; Axis I: Scene Understanding (0–10); Axis II: Turn & Speaker Selection Discipline (0–10); Axis III: Role Introduction & Utilization Judgment (0–10); Axis IV: Overall Assessment (0–10)
- Scene Manager evaluation ignores prose quality and judges system-level orchestration on scene state, fair turn order, role introductions, and holistic pacing and user agency.Its conservative 0–10 rubric checks causally justified scene transitions, speaker validity and user inclusion, appropriate role timing and utility, and caps overall scores for major axis failures.Source: §F.2 · p. 25, Axis I: Scene Understanding (0–10) · p. 25, Axis II: Turn & Speaker Selection… · p. 25
G Training Details; H Basic vs Enhance for Actor and Scene Manager
- Training uses a 5% validation holdout, 8 epochs, a fixed 1e−6 learning rate, and a maximum sequence length of 16K tokens.The appendix also provides Basic and Enhance prompt variants: Actor prompts differ in role-playing annotations and adaptive constraints, while Scene Manager prompts differ in action-selection guidance.Source: G Training Details · pp. 25–26, H Basic vs Enhance for Actor and Scene… · p. 26
I Additional Evaluation Frameworks; I.1 CharacterArena Evaluation; I.2 CharacterBench Evaluation
- CharacterArena compares model trajectories by judged pairwise win rate, while CharacterBench evaluates single-turn responses against profile, context, and user query.CharacterArena uses six dimensions including plot development, immersion, interactivity, and coherence; CharacterBench’s 13 metrics across six aspects cover memory, knowledge, persona, emotion, morality, and believability, testing transfer beyond AdaMARP’s simulation setting.Source: I Additional Evaluation Frameworks · p. 26, §I.1 · p. 26, §I.2 · pp. 26–27
I.3 Human Evaluation · J Results on Additional Evaluation Frameworks
On AdaptiveBench, human evaluators preferred Qwen2.5-7B-Instruct trained with AdaRPSet, with an average win rate of 80.33% and variance of 6.22; this was below the 94% AI-based CharacterArena result, but the trend was consistent.
I.3 Human Evaluation
- Because annotators could not reliably score all 12 fine-grained dimensions, human evaluation used pairwise preferences on AdaptiveBench.Evaluators compared two trajectories and chose which actor better portrayed the main character.Source: §I.3 · p. 27
- Evaluators independently compared trajectories generated under the same Profiles, Motivations, and initial scenario.They followed the official evaluation guidelines of the judge model.Source: §I.3 · p. 27
- Three independent evaluators were treated as separate samples, with individual win rates summarized by their mean and variance rather than majority vote.Source: §I.3 · p. 27
- Crab, CoSER, and BeyondDialogue were excluded because their missing thought or environment fields, or speech-only outputs, made their generating models identifiable.Source: §I.3 · p. 27
- The human study controlled its comparison to Qwen2.5-7B-Instruct before and after training with AdaRPSet.Source: §I.3 · p. 27
- 80.33% average win rate (variance 6.22) favored Qwen2.5-7B-Instruct after AdaRPSet training, below the 94% AI-based CharacterArena result but consistent in direction.The 94% result was obtained using CharacterArena’s official prompts.Source: §I.3 · p. 27
J.1 CharacterArena Results · J.2 CharacterBench Results
AdaMARP outperforms baselines in CharacterArena evaluations and achieves the highest average CharacterBench score for Qwen2.5-7B-Instruct. For Llama-3.1-8B, results are mixed on CharacterBench, where adding general instruction data improves its average score and slightly reduces AdaptiveBench performance.
J.1 CharacterArena Results
- 94% vs Base, 100% vs BeyondDialogue, 99% vs CoSER, and 84% vs Crab are Ours’ CharacterArena win rates for Qwen2.5-7B-Instruct.The results support superior trajectory-level performance under independent pairwise judging.Source: §J.1 · pp. 27–28, Table 6 · p. 28
- Ours significantly outperforms Crab and CoSER on CharacterArena with Llama-3.1-8B; BeyondDialogue is excluded because its small training dataset impaired instruction following.The excluded BeyondDialogue variant was trained on approximately 3k dialogues, including approximately 800 English dialogues.Source: Table 7 · p. 28, §J.1 · p. 28
J.2 CharacterBench Results
- 3.68 is Ours’ highest average CharacterBench score for Qwen2.5-7B-Instruct, with first- or second-place ranks on most critical dimensions.These dimensions include Attribute Consistency (Human), Behavior Consistency (Bot), Human-likeness, and Engagement.Source: §J.2 · p. 28, Table 8 · p. 29
- 3.52 vs 3.53 is the CharacterBench overall average for Llama-3.1-8B-Ours vs CoSER, while Crab achieves the best results.The passage attributes this gap primarily to general instruction-following data in Crab and CoSER that is absent from AdaRPSet.Source: §J.2 · p. 28, Table 8 · p. 29
- +0.05 is the overall CharacterBench average-score improvement for Llama-3.1-8B-Ours+General after adding 22,425 Tulu3 instruction-tuning instances.The added instruction data matches AdaRPSet in size.Source: §J.2 · p. 28, Table 8 · p. 29
- 8.89 → 8.80 is the AdaptiveBench performance change for Llama-3.1-8B-Ours+General, which improves Instruction Compliance by +0.17 and still beats competing methods.The added general instruction data improves Instruction Compliance while causing minor declines on several other dimensions.Source: §J.2 · p. 28, Table 9 · p. 29
K Ablation Analysis of Actor-Model Evaluation
Actor-Model evaluation rankings on AdaptiveBench are largely consistent across judge models, with GPT-4o-mini producing some middle-tier differences; GPT-5-Chat is used as the default judge.
K Ablation Analysis of Actor-Model Evaluation
- The robustness study scores AdaptiveBench trajectories with GPT-4o-mini, GPT-5-Chat, Gemini-2.5-Pro, and Doubao-1.5-Pro-Character under the same protocol and prompts.Rankings are summarized under Basic and Enhance prompting variants.Source: K Ablation Analysis of Actor-Model… · p. 29
- Three judges—GPT-5-Chat, Gemini-2.5-Pro, and Doubao-1.5-Pro-Character—produce almost identical model rankings, while GPT-4o-mini differs in some middle-tier ordering.This indicates that the evaluation conclusions are not sensitive to a particular judge choice.Source: K Ablation Analysis of Actor-Model… · p. 29
- GPT-5-Chat is adopted as the main paper’s default judge because its rankings align with other strong judges and provide a unified evaluation standard.Source: K Ablation Analysis of Actor-Model… · p. 29
L Ablation Analysis of Scene-Manager Evaluation
Across both Scene Manager prompting variants, changing the judge yields stable rankings, with Claude Sonnet 4.5 or its Basic variant ranked first and Doubao-1.5-Pro-Character last; GPT-5-Chat is selected as the default judge.
Ablation Analysis of Scene-Manager Evaluation
- The ablation swaps among five judges while holding evaluated trajectories and the scoring rubric fixed, and tests both Basic and Enhance Scene Manager prompting variants.Judges are GPT-4o-mini, GPT-5-Chat, Doubao-1.5-Pro-Character, Gemini-2.5-Pro, and Claude Sonnet 4.5.Source: L Ablation Analysis of Scene-Manager… · p. 29
- Claude Sonnet 4.5 (or its Basic variant) ranks first and Doubao-1.5-Pro-Character last; Doubao judging occasionally swaps GPT-4o-mini and GPT-5-Chat, while top and bottom positions remain unchanged.The overall ranking trend remains stable across judges.Source: L Ablation Analysis of Scene-Manager… · pp. 29–30
- GPT-5-Chat is used as the default Scene-Manager evaluation judge to provide a unified evaluation standard.The authors make this choice based on the observed robustness of the rankings.Source: L Ablation Analysis of Scene-Manager… · p. 30
M Prompting Strategy Analysis · N Token Consumption and Cost Analysis
Prompting effects diverge by role: Basic often performs slightly better for the Actor Model, while the Scene Manager generally benefits from Enhance. The appendix also examines token consumption and computational resources across three phases.
M.1 Actor-Model Prompting Strategy
- The Actor-Model comparison evaluates concise Basic and more constrained Enhance prompts under different judge models.Despite the observed trend, the main experiments use Enhance by default for a unified, conservative evaluation setting; unless noted, reported Actor-Model results use it.Source: §M.1 · pp. 30–31
- Across judge models, the Actor Model’s Basic prompt often achieves slightly better overall performance than Enhance.The analysis suggests overly restrictive instructions can reduce flexibility on AdaptiveBench.Source: §M.1 · pp. 30–31
- The lightweight Basic prompt leaves the Actor Model more freedom to integrate Thought–Action–Speech–Env signals contextually.By contrast, the more constrained Enhance prompt may hinder responses to role shifts, scene transitions, and multi-party interactions.Source: §M.1 · pp. 30–31
M.2 Scene-Manager Prompting Strategy
- Scene-Manager Enhance provides finer-grained guidance on scene understanding, speaker selection discipline, and role-introduction judgment.The main experiments adopt Enhance as the Scene Manager’s default prompt unless otherwise specified.Source: §M.2 · p. 31
- Across most judge models, Scene-Manager Enhance yields equal or higher Overall Assessment than Basic.Explicit, structured instructions can reduce ambiguity and improve consistency in global scene state, turn-taking, and role-introduction decisions.Source: §M.2 · p. 31
N Token Consumption and Cost Analysis
- The token-consumption and computational-resource analysis covers data construction, trajectory generation, and evaluation.The supplied passage identifies these analysis phases but reports no cost or token figures.Source: N Token Consumption and Cost Analysis · p. 31
N.1 Data Construction Cost … O Case Study
The appendix reports costs for synthetic-data construction, AdaptiveBench trajectory generation, and trajectory evaluation, then compares structured role-playing outputs qualitatively. It identifies the Scene Manager as the largest generation-cost component and notes channel-confusion errors in CoSER outputs.
N.1 Data Construction Cost
- GPT-5-Chat synthesis of AdaRPSet-Synthesis cost approximately $168 USD for 1,000 trajectories spanning 20 topics with 50 instances each.The cost covers generation of the complete synthetic set of complex, multi-turn adaptive plots.Source: §N.1 · p. 31
N.2 Trajectory Generation in AdaptiveBench
- A single 100-dialogue AdaptiveBench run orchestrates three entities: the Scene Manager, Actor Model, and User Model.The cost accounting covers generation and evaluation across the simulated dialogues.Source: §N.2 · p. 33
- For 100 AdaptiveBench trajectories, the Scene Manager processes 7.5M input tokens and generates 150k output tokens, making it the highest-cost generation component.Each trajectory requires approximately 75,000 input tokens and 1,500 output tokens for the Scene Manager.Source: §N.2 · p. 33
- For each of the Actor and User Models, 100 trajectories require approximately 1M input tokens and 50k output tokens, with nearly identical overhead for both.Each entity averages about 10,000 cumulative input tokens and 500 output tokens per dialogue, generating only when selected by the manager.Source: §N.2 · p. 33
N.3 Evaluation of Actor and Scene Manager
- Actor Model judging of 100 trajectories uses 500k input tokens and 100k output tokens in a one-pass evaluation of each trajectory’s full context.The judge averages ∼5,000 input tokens and ∼1,000 output tokens per trajectory.Source: §N.3 · p. 33
- Scene Manager evaluation of 100 trajectories uses roughly 450k input tokens and 50k output tokens, an overhead of the same order as Actor Model evaluation.Each dialogue requires approximately 4,000–5,000 input tokens and 400–500 output tokens.Source: §N.3 · p. 33
O Case Study
- The case study compares structured role enactment by Crab, CoSER, and Ours, while excluding Beyond because it produces only Speech without explicit Action or Thought annotations.Crab generates Speech and Action; CoSER generates Speech, Action, and Thought; Ours supports all required channels, including Environment.Source: O Case Study · p. 35
- CoSER’s environment descriptions include tag misuse and modality confusion: it conflates Thought and Action in Turns 3 and 6 and embeds environmental descriptions in Thought in Turn 5.These errors reduce the reliability of its structured outputs despite its ability to produce environment-related descriptions.Source: O Case Study · p. 35
O.1 Character Consistency … O.5 Instruction Compliance
The case study presents AdaMARP’s role-playing advantage as alignment among character psychology, embodied action, environmental grounding, and long-horizon narrative control, supporting faithful and sustainable role-playing.
O.1 Character Consistency
- Amaris’s internal hesitation motivates a subtle withdrawal and restrained speech, maintaining a coherent causal loop that reflects her core motivation.The passage contrasts this tri-modal alignment with Crab’s collapsed loop and CoSER’s explicit emotional conclusions.Source: §O.1 · p. 35
- Amaris’s stable speaking style conveys guardedness through restraint, irony, and craft-oriented deflection rather than explicit emotional expression.The passage contrasts this subtlety with CoSER’s direct explanations and Crab’s flattening style.Source: §O.1 · p. 36
O.2 Environmental Grounding
- Environmental cues shape character cognition and decisions, while events such as a door knock ground Inaya’s entrance and preserve scene continuity.CoSER’s environmental descriptions often do not affect decisions, Crab underuses the environment, and both systems have less grounded transitions.Source: §O.2 · pp. 36–37
O.3 Interpersonal Interaction
- Amaris responds to Cassian’s emotional subtext while maintaining relational tension, and the model keeps Inaya’s catalytic role from overriding Amaris’s agency.By comparison, Crab often resolves threads too quickly or reduces relational nuance, while CoSER may shift registers or let Inaya overstep.Source: §O.3 · p. 37
O.4 Narrative Progression
- The model advances the narrative through incremental openings and deferred resolution, sustaining tension while preserving character motivations, relationships, and environmental facts across extended interaction.Crab tends toward early convergence, whereas CoSER can cycle through hesitation without adding narrative affordances.Source: §O.4 · p. 37
O.5 Instruction Compliance
- The model follows role-playing protocol constraints, respecting speaker boundaries, correctly scoping Thought, Action, and Environment annotations, and avoiding content on behalf of the user or scene manager.Source: §O.5 · p. 37
- The case study attributes more faithful and sustainable role-playing to tighter alignment among character psychology, embodied action, environmental grounding, and long-horizon narrative control.It says the advantage does not come from more dramatic or emotionally explicit dialogue.Source: §O.5 · p. 37