Source-linked AI summary
Deflanderization for Game Dialogue: Balancing Character Authenticity with Task Execution in LLM-based NPCs
Pasin Buakhaw, Kun Kerdthaisong, Phuree Phenhiran, Pitikorn Khlaisamniang, Supasate Vorathammathorn, Piyalitt Ittichaiwong, Nutchanon Yongsatianchot
TL;DR
Video-game NPCs must combine task execution with persona-consistent dialogue, yet maintaining consistent and deep personas over long interactions is challenging. The paper examines prompting, retrieval, and fine-tuning approaches in CPDC 2025. Its findings report complementary strengths across API prompting and GPU fine-tuning, alongside an API Task 3 score gain for Deflanderization.
Problem
LLM research in entertainment, particularly video games, remains relatively underexplored, and maintaining dynamic persona consistency and depth over long interactions is challenging.
Source: §1 · p. 2Method
The paper studies Deflanderization prompting, retrieval-augmented memory, and SFT with LoRA as complementary strategies for persona-consistent, context-aware, goal-directed NPCs.
Source: §1 · p. 2Results
+0.013 absolute gain in CPDCscore (all) on Task 3 for D-RW versus zero-shot; overall, prompting-based Deflanderization helped in API settings while fine-tuned large models performed strongly in the GPU track.
Source: §5.1 · p. 5, §6 · p. 6Takeaways & Limitations
The findings identify balancing persona consistency with functional precision as a challenge and point to hybrid prompting with retrieval-augmented fine-tuning as a direction for future work.
Source: §6 · p. 6
Abstract
from arXiv · showhide
The emergence of large language models (LLMs) has opened new opportunities for creating dynamic non-player characters (NPCs) in gaming environments, enabling both functional task execution and persona-consistent dialogue generation. In this paper, we (Tu_Character_lab) report our participation in the Commonsense Persona-Grounded Dialogue Challenge (CPDC) 2025 Round 2, which evaluates agents across three tracks: task-oriented dialogue, context-aware dialogue, and their integration. Our approach combines two complementary strategies: (i) lightweight prompting techniques in the API track, including a Deflanderization prompting method to suppress excessive role-play and improve task fidelity, and (ii) fine-tuned large models in the GPU track, leveraging Qwen3-14B with supervisedfinetuning (SFT) and Low-Rank Adaptation(LoRA). Our best submissions ranked 2nd on Task 1, 2nd on Task 3 (API track), and 4th on Task 3 (GPU track).
1 Introduction
The introduction frames game NPCs as a setting where dynamic dialogue must coexist with task execution and persistent character consistency. It presents a combined prompting, retrieval, and fine-tuning approach for the CPDC 2025 challenge.
1 Introduction
- Video games remain relatively underexplored in LLM research, while conventional development relies heavily on predetermined scripts and dialogue trees.LLMs are being incorporated into NPCs to support dynamic, context-aware conversations.Source: §1 · p. 2
- The introduction describes long-term consistency and depth of dynamic personas as a challenge, with flanderization reducing a complex character to a caricature of one exaggerated trait.The paper relates the term to Ned Flanders becoming a one-dimensional religious zealot.Source: §1 · p. 2
- The CPDC 2025 challenge calls for fantasy-RPG NPC agents that demonstrate both persona consistency and task execution proficiency.The introduction situates the work within a competition focused on both capabilities.Source: §1 · p. 2
- The paper combines Deflanderization prompting, retrieval from prior interactions, and SFT with LoRA to study persona-consistent, context-aware, goal-directed NPC performance.The retrieval mechanism draws on memory and similarity-based retrieval to align responses with in-game history and world knowledge.Source: §1 · p. 2
- The paper identifies maintaining dialogue generation and function-generation ability, and examining their performance trade-offs, as goals for its Deflanderization prompting technique.Its participation spans the competition's API and GPU divisions.Source: §1 · pp. 2–3
2 Related Work
Related work covers game-oriented dialogue agents and tool-calling systems, including the challenge of evaluating agents in collaborative settings where users and agents can both act.
2.1 Agents for Game-Oriented Dialogue
- Game-oriented dialogue research includes assistants for game questions and systems using dialogue and goal-verifying agents with shared memory to align actions with task goals.The cited game assistant helps users navigate game mechanics; the two-agent framework aligns dialogue and actions with overarching goals.Source: §2.1 · p. 3
2.2 Tool calling
- Tool-calling systems use LLMs to access external functions or resources, and commonly combine action execution, environment perception, validation, planning, and tool retrieval (Xu et al., 2025).The cited passage presents robust evaluation benchmarks as a key challenge in this domain.Source: §2.2 · p. 3
- Tool-calling benchmarks include dual-control τ 2-Bench, where both user and agent act in a shared dynamic world; reported performance degrades significantly in this setting.The passage identifies coordination and communication challenges absent from single-user-control evaluations.Source: §2.2 · p. 3
3 Competition Overview
The CPDC competition evaluates agents in fantasy-RPG dialogue settings across API and GPU tracks, with tasks spanning task-oriented dialogue, context-aware dialogue, and their integration.
3.1 Competition Tasks
- Each CPDC track includes task-oriented dialogue, context-aware dialogue, and an integrated task combining both capabilities.The competition aims to combine contextual understanding, knowledge use, and task execution.Source: §3.1 · p. 3
3.1.1 Task 1: Task-Oriented Dialogue Agents
- Task 1 agents assess conversation context, select needed function calls, and provide aligned arguments; evaluation emphasizes call and argument correctness.The passage gives merchant weapon selection as an example of function use.Source: §3.1.1 · p. 3
3.1.2 Task 2: Context-Aware Dialogue Agents
- Task 2 agents generate NPC responses aligned with assigned personas, and evaluation emphasizes consistency with defined character traits.Source: §3.1.2 · p. 4
4 Competition Tracks
The API track explores prompting methods, while the GPU track combines retrieval-based grounding with fine-tuned large models. Their reported submissions achieved placements across integrated and individual tasks.
4.1 API Track
- The API track explored prompting with GPT-4o-mini; Deflanderization asks for natural, concise replies without exaggerated role-play, while RW removes world-setting information.The authors' error analysis found baseline responses could overemphasize narrative persona instead of the player's immediate intent and appropriate function use.Source: §4.1 · p. 4
- The API submission using D-RW with two turns of sample dialogue ranked 2nd on Task 3, 2nd on Task 1, and 5th on Task 2.Source: §4.1 · p. 4
4.2 GPU Track
- The GPU track used hybrid RAG and memory to retrieve prior interactions for function selection and dialogue drafting, and explored refining drafts against similar golden responses.Source: §4.2 · p. 5
- The GPU track's best-performing Qwen3-14B submission ranked 4th on Task 3 after full SFT followed by LoRA-SFT on dialogue and function-calling data.The LoRA-SFT stage used rank 32 and α = 32.Source: §4.2 · p. 5
5 Results
In the API track, Deflanderization improved scores over zero-shot, few-shot examples further benefited Task 1, and more complex prompting produced marginal or inconsistent gains. In the GPU track, Qwen3-14B with SFT and LoRA outperformed smaller baselines, while retrieval methods showed modest Task 1 gains and did not match LoRA-SFT’s joint-task result.
5.1 API Track
- Adding few-shot examples to Deflanderization improved Task 1 by +0.092 and +0.133 on train.json, respectively.The best API submission combined D-RW with two-turn examples and ranked 2nd on Task 3, 2nd on Task 1, and 5th on Task 2.Source: §5.1 · p. 5, Table 1 · p. 6
- CoT, guiding responses, and Most Word constraints yielded marginal or inconsistent gains; CoT raised Task 1 BERTScore but reduced function argument accuracy.Most Word improved BLEU on train.json but did not transfer to leaderboard CPDCscore; verbose reasoning may dilute key arguments.Source: §5.1 · p. 5
- Qwen3-14B with full SFT and LoRA reached 0.598 all-score and ranked 4th, while smaller baseline models scored below 0.40.The passage attributes this improvement to both model size and targeted finetuning on domain-specific data.Source: §5.1 · p. 5
- RAG+Refine and RAG+Memory raised Qwen3-8B performance to 0.522 for Task 1, but retrieval gains fell short of those from LoRA-SFT.The authors cite limited retrieval-corpus scale and the difficulty of adding retrieved context without overloading prompts.Source: §5.1 · pp. 5–6
- D-RW gained +0.013 absolute CPDCscore (all) over zero-shot on Task 3, and Deflanderization improved scores across both tasks.Table 2 identifies the Task 3 API-track results; the passage attributes the gain to avoiding excessive role-play that can divert attention from functional correctness.Source: §5.1 · p. 5, Table 2 · p. 6
- LoRA-SFT produced the highest joint score across Tasks 1 and 2, while RAG+Refine’s Task 1 score of 0.522 came with weaker Task 2 performance.The comparison suggests that alignment across functional reasoning and persona-grounded dialogue requires joint optimization rather than isolated improvements.Source: §5.1 · p. 6
6 Discussion
The discussion presents prompting and finetuning as complementary strategies across the API and GPU tracks. It identifies balancing persona consistency with functional precision as a challenge and proposes hybrid prompting and retrieval-augmented finetuning for future work.
6 Discussion
- Prompting-based Deflanderization with few-shot grounding worked in low-resource API settings, while finetuned large models performed best in the GPU track.The discussion characterizes the approaches as complementary within the CPDC 2025 setting.Source: §6 · p. 6
- Across both tracks, methods that improved role-play fidelity sometimes hurt argument correctness, and vice versa.The authors identify balancing persona consistency with functional precision as a continuing challenge.Source: §6 · p. 6
- The authors propose exploring hybrid strategies that combine lightweight prompting with retrieval-augmented finetuning for fantasy RPG agents.The stated aim is to sustain both accuracy and believability.Source: §6 · p. 6
A Exploratory Data Analysis
The exploratory analysis describes character demographics and event timing in the Task 2 training data, as well as the return-value ratio figure for guild NPC responses in Task 1.
A Exploratory Data Analysis
- The Task 2 training data contains 20 merchants and 20 guild receptionists; most NPCs are women, who are younger than the men.Figure 3 presents the age-gender distribution.Source: Figure 3 · p. 9
- Most Task 2 training events occur after 1 p.m., and winter events occur only at the quest reception place.Figure 4 describes the date-time distribution.Source: Figure 4 · p. 9
- Figure 5 encodes guild NPC response return-value ratios, with green for return and red for no return.The supplied passage gives the figure’s encoding but no ratio values.Source: Figure 5 · p. 9
B Evaluation Metrics
The paper uses selected dialogue-generation metrics in local experiments but notes that automatic metrics alone are not fully reliable, so final private-leaderboard evaluation used human evaluation.
B Evaluation Metrics
- CPDC2023 used WordF1, BLEU, CPDScore, USEScore, and BERTScore for automatic dialogue-generation evaluation.The passage introduces these as metrics the authors tried in their local environment.Source: B Evaluation Metrics · p. 9
- Figure 6 is titled as merchant NPC response return-value ratios, while Figure 7 maps merchant and guild receptionist frequency to weather.The supplied figure descriptions do not state any comparison outcome.Source: Figure 6 · p. 9, Figure 7 · p. 9
- Automatic metrics alone are not fully reliable for evaluating dialogue systems, so organizers used human evaluation for the final private leaderboard.The limitation is stated with citations to Liu et al. (2016) and Novikova et al. (2017).Source: B Evaluation Metrics · p. 9
B.1 Task 1
Task 1 evaluates function-call accuracy through exact matches on function names and arguments, and semantic similarity through BERTScore. These metrics assess different aspects of generated function calls.
B.1.1 Function name exact match
- Exact function-name match checks whether each predicted function name matches its reference.The passage also identifies the compared names as the function-name sets for each instance.Source: §B.1.1 · p. 9
B.1.2 Function argument exact match
- Exact argument match checks whether all predicted arguments match the reference.The compared argument sets are defined for each instance.Source: §B.1.2 · p. 10
B.1.3 BERTScore
- BERTScore measures the semantic similarity of function calls using prediction and reference tokens.The passage describes token sequences from the prediction and reference as inputs.Source: §B.1.3 · p. 10
B.2 Task 2
Task 2 evaluates role-playing consistency with dialogue metrics and uses prompting and generated examples to shape NPC dialogue and function calls. The section describes the scoring components and several API-track prompt variants.
B.2 Task 2; B.2.1 BLEU-4
- Task 2 role-playing consistency is evaluated with four metrics, including BERTScore; BLEU-4 combines modified n-gram precision with a brevity penalty.The brevity penalty is 1 when candidate length exceeds reference length, and otherwise exp(1−r/c).Source: §B.2 · p. 10, §B.2.1 · p. 10
B.2.2 Word-level F1
- Word-level F1 is calculated over token sets after tokenizing the prediction and reference with NLTK.The section names the token sets as Tpred and Tref.Source: §B.2.2 · p. 10
B.2.3 CPDCscore
- CPDCscore weights WordF1, BLEU, USEScore, and BERTScore for dialogue, and weighted exact match for function names and arguments.The passage describes these as the leaderboard's expected weighting.Source: §B.2.3 · p. 10
C.1 Additional Data Generation
- Additional data generation covers merchant function-call examples grounded in supplied knowledge and Task 2 dialogue with clear in-world purpose and concise NPC replies.The merchant prompt requests player dialogue and a function call with parameters; the dialogue prompt calls for short, natural responses that avoid exaggerated role-play.Source: §C.1 · pp. 10–11
C.2 FewShot
- Few-shot prompts provide Task 2 dialogue examples and Task 1 instructions for selecting functions and filling their arguments from the supplied information.The Task 1 prompt allows multiple function calls or none.Source: §C.2 · pp. 11–12
C.3 Chain of Thought
- The Task 1 Chain-of-Thought prompt requires a reasoning paragraph before function calls, even when the reasoning seems obvious.The prompt specifies a fixed reasoning format and permits multiple or no function calls.Source: §C.3 · p. 12
C.4 Deflanderization
- The Deflanderization Task 2 prompt asks for concise, natural NPC responses based on supplied knowledge, avoiding exaggerated role-play or guessing.It provides function-call and general knowledge alongside the character profile and worldview.Source: §C.4 · p. 12
C.5 Most word
- The Most word prompt guides NPCs toward colloquial alternatives to formal phrasing, including simpler expressions of caution, agreement, and uncertainty.Its examples contrast conversational phrases with more formal or dramatic alternatives.Source: §C.5 · p. 13
D Compute Constraints
The compute constraints specify separate hardware and runtime limits for the GPU and API tracks. API use is restricted by call, token, model, and network-access conditions.
GPU Track
- The GPU track used an AWS g6e.2xlarge node with 8 vCPUs, 64 GB RAM, and an L40s GPU with 48 GB VRAM.The per-turn timeout was 7 seconds.Source: D Compute Constraints · p. 14
API Track
- The API track used an AWS m5.large node with 2 vCPUs and 8 GB RAM, with at most two API calls per utterance.Each turn had a 2,000-token input limit and a 200-token output limit.Source: D Compute Constraints · p. 14
- The API track allowed only GPT-4o-mini, prohibited fine-tuned API models, and expected network access to be blocked.These are server and access constraints for API usage.Source: D Compute Constraints · p. 14
E Additional Results
The additional results report Qwen3-8B scores across three tasks and describe SFT, LoRA, GRPO, and inference configurations. The final API leaderboard table distinguishes human-evaluated Tasks 2 and 3 from automatically evaluated Task 1.
E Additional Results
- Task 3 achieved a CPDCScore of 0.324, Task 1 achieved 0.290, and Task 2 achieved 0.359 with Qwen3-8B tuned using SFT with LoRA and GRPO.The passage reports these as resulting task scores.Source: E Additional Results · p. 14
E.1 Supervised Fine-Tuning (SFT)
- Task 2 SFT used the original dataset and additional generated samples, with training implemented in Unsloth for up to 30 steps.The section gives gradient accumulation steps as 1.Source: §E.1 · p. 14
E.2 LoRA
- Task 1 used LoRA in combination with SFT on the dataset.The passage introduces the main configuration but does not specify its settings here.Source: §E.2 · p. 14
E.3 GRPO Tuning on Reasoning Data
- GRPO tuning used curated enchanted reasoning interactions with persona metadata such as background, personality traits, and goals.An example pairs a player–NPC role-play with reasoning about inferred urgency and persona faithfulness.Source: §E.3 · pp. 14–15
E.4 Inference with vLLM and LoRA Adapters
- Inference used vLLM to serve the base model and LoRA-tuned function-generation checkpoints, with dynamically loaded adapters.The listed configuration includes a 4096-token maximum sequence length, 0.5 GPU memory utilization, and maximum LoRA rank of 64.Source: §E.4 · p. 15
F Final Leader Board
- The final API leaderboard table reports team results, with Tasks 2 and 3 evaluated by humans and Task 1 evaluated automatically.The supplied passage identifies the table as the team's final API-track result but provides no cell values.Source: Table 4 · p. 16