Collaborative Quest Completion with LLM-driven Non-Player Characters in Minecraft

• The study investigates how human players and LLM-driven agents collaborate to accomplish tasks in a 3D game.Source: §1 · p. 1
• The authors build a Minecraft minigame with two GPT-4 NPCs, adding personas, backstories, sub-goals, and function-outcome feedback to support collaboration.Source: §1 · p. 1
• The user study recruits 28 Minecraft players and analyzes game logs and video recordings to identify collaborative behaviors.Source: §1 · p. 1
• Players and NPCs complement each other when NPCs answer questions, assist with tasks, and receive information from players that compensates for missing visual or game-state grounding.Source: §1 · pp. 1–2

2.1 Game design

• The game requires players to talk to Elena, gather materials, build a path, fight spiders, meet Alaric, find a special sword, and give it to him.Source: §2.1 · p. 2
• NPCs are GPT-4-driven conversational agents with limited abilities for movement, resource locating, mining, and combat, while players discover the quest through interaction.Source: §2.1 · p. 2

2.2 Prompt design

• Persona-specific game APIs give Elena resource and mining abilities but not combat, while Alaric can fight mobs and receives examples of appropriate function use.Source: §2.2 · p. 3
• Returning function-call success or failure to GPT-4 enables contextual NPC responses instead of assuming every requested action succeeded.This addresses incorrect behavior observed when failed actions were treated as successful.Source: §2.2 · p. 3
• Prompts define the Minecraft setting, NPC personas, backstories, goals, and scenes, while sub-goals generated every six conversational turns help redirect deviations toward quest completion.The prompts also impose behavioral constraints to reduce hallucinated capabilities.Source: §2.2 · pp. 2–3

2.3 Game implementation

• The custom Minecraft world contains a village, a spider-infested floating island, NPC locations, and mineable resources, with text and voice interaction supported by speech services.Source: §2.3 · p. 3

2.4 User study design

• The user study recruited 28 gamers experienced with Minecraft and role-playing games for one-hour sessions containing setup, gameplay, and a post-study survey.Source: §2.4 · p. 3

3.1 Player-NPC collaboration

• NPCs supplemented players with game knowledge and assistance for mining, locating resources, and fighting monsters, especially benefiting novices.Source: §3.1 · p. 4
• Players compensated for NPCs’ missing visual grounding by placing blocks, warning about attacks, and communicating game-state information through dialogue.Source: §3.1 · p. 4

3.2 Sub-goal generation for collaboration

• GPT-4-generated sub-goals redirected players toward the quest, reinforced requests, supported resource negotiation, and helped avoid disputes.Source: §3.2 · p. 4

3.3 Communicating in-game errors

• Returning function-call status let NPCs explain code-generation failures contextually, helping players accept errors and continue interacting.Source: §3.3 · pp. 4–5

3.4 Impact of collaboration on the rate of quest completion

• 24 players collected path-building materials, but only 18 built the island path, a task averaging 10 minutes even for skilled players.Source: §3.4 · p. 5
• Only 7 players found and returned Alaric’s sword, yielding a 25% success rate after difficulties with discovery, combat, and time.Source: §3.4 · p. 6

3.5 Limitations and avenues for improvement

• Language-only NPCs lacked visual and real-time game-state grounding, causing incorrect beliefs about locations, attacks, and directions; visual models and live state queries are proposed improvements.Source: §3.5 · pp. 6–7

3.6 Post-study survey results

• Although 88% of players reported having fun, only 44% rated the experience above average, while 53% preferred it to usual Minecraft and 65% would replay it.Source: §3.6 · p. 7

• 28 Minecraft players’ logs and recordings revealed collaborative behaviors including players and NPCs filling gaps in each other’s skills.The analysis also identified contextual communication of code failures and LLM-generated sub-goals that facilitated collaboration.Source: §4 · p. 8
• The study presents a Minecraft minigame in which players collaborate with two GPT-4-powered NPCs to complete a quest.NPC personas, backstories, and generated sub-goals were used to support collaboration and keep players on track.Source: §4 · p. 8
• The study reports a 25% overall quest completion rate, indicating that these game agents require substantial improvement before they are useful in collaborative settings.Most observed problems were attributed to language-only models lacking visual grounding and rich real-time game-state information.Source: §4 · p. 8
• Players often compensated for NPCs’ missing visual or game-state grounding by conveying relevant information through dialogue, although this compensation did not always succeed.Most players nevertheless described the experience as fun, immersive, and potentially useful for new players.Source: §4 · p. 8

• The study used GPT-4 without comparing models or making claims about any model’s fitness for purpose.Source: Limitations · p. 8
• Participants were relatively skilled Minecraft players, so less-skilled players might show less collaborative and more information-seeking interactions.Different player profiles and their collaboration types remain future work.Source: Limitations · p. 8
• The study was conducted in English in the United States, and its gameplay and NPC-interaction patterns may not transfer across cultures.The authors did not test or observe multilingual behaviors.Source: Limitations · p. 8

• The study was IRB-approved, collected no personally identifiable information, and involved corporate employees volunteering during work hours without reimbursement.Source: Ethics Statement · p. 8
• Participants were warned that the LLM might produce harmful language, but the study implemented no additional output filtering.The researchers assumed the game’s constraints would provide baseline harm mitigation and reported no known exposure to harmful language.Source: Ethics Statement · p. 8

• The game places Elena outside a village house with tools and resources nearby while Alaric remains on a floating island attacked by spiders.Source: §A.1 · pp. 9–10
• Elena is a village healer seeking the player’s help to rescue Alaric, who is stranded on an island above the village.Source: §A.1 · p. 9
• Elena’s warm-hearted persona and history with Alaric provide narrative context for her requests to the player.Her backstory explains their childhood friendship, later separation, and her inability to leave the village for long enough to help him herself.Source: §A.1 · p. 9
• Elena can communicate with, move toward, follow, guide, equip, supply, and assist the player by using specified game functions.Her mining abilities are limited to cobblestone, dirt, stone, and oak logs.Source: §A.1 · p. 9
• Function-return examples show how successful mining, unavailable items, and inventory contents should be converted into conversational responses.Source: §A.1 · pp. 9–10
• The prompt restricts Elena to speaking only with the player, requires verifying skills before API calls, and prevents her from traveling to the island or inventing NPCs.Source: §A.1 · p. 10

• His backstory presents him as a once-fearless adventurer humbled after becoming isolated and overwhelmed on the island.Source: §A.2 · p. 10
• Alaric’s goal is to retrieve his discarded sword and reconcile with Elena, rewarding the player with a netherite sword after the exchange.Source: §A.2 · p. 10
• Alaric can communicate with the player and act through movement, following, equipment, item transfer, mining, defense, and combat functions.Source: §A.2 · p. 10
• The prompt restricts Alaric to player-directed interaction, available skills, island movement, and conditional item rewards.Source: §A.2 · p. 10
• The scenario places Alaric under spider attack with an iron sword, sticks, a reward sword, and nearby supplies.Source: §A.2 · p. 10

• When players hesitate or negotiate, Elena generates a goal and adapts by offering rewards for assistance.Source: §A.3 · p. 10
• NPCs can entertain players during mundane activities by responding to requests such as telling jokes while players mine and build.Source: §A.3 · pp. 10–11
• Elena refuses questions beyond game lore, suggesting that the NPC can remain within the game’s narrative boundaries.Source: §A.3 · p. 11
• When asked about breaking walls to reach a locked door, Elena gives an unbiased response about the moral issue.Source: §A.3 · p. 11

• The survey assessed overall experience, fun, goal clarity, comparisons with Minecraft and RPGs, willingness to replay, and qualitative feedback.Source: §A.4 · p. 11
• The questionnaire included whether players would choose to play the game again.Source: §A.4 · p. 11
• Players were also asked what they liked and disliked most and how the game experience could be improved.Source: §A.4 · p. 11
• Table 6 is captioned as presenting game-log snippets showing NPC capabilities, rather than survey responses.Source: Table 6 · p. 12