PaperlayerView exact revision on arXiv

Source-linked AI summary

A Survey on Large Language Model-Based Game Agents

Sihao Hu, Tiansheng Huang, Gaowen Liu, Ramana Rao Kompella, Fatih Ilhan, Selim Furkan Tekin, Yichang Xu, Zachary Yahn, Ling Liu

arXiv:2404.02039v5cs.AI

TL;DR

LLMs are largely optimized on static text, while interactive intelligence involves environmental perception and action; games offer a controllable setting for studying that interaction. This survey organizes LLM-based game-agent research with single- and multi-agent frameworks and a genre-linked challenge taxonomy. It synthesizes design patterns and open challenges across these perspectives.

  • Problem

    LLMs are primarily optimized on static text, while embodied intelligence develops through ongoing interaction with an environment, and prior surveys leave LLM-based game agents underexplored as a distinct area.

    Source: §1 · p. 1

  • Method

    The survey reviews LLM-based game agents through a unified architecture for memory, reasoning, perception-action, communication, and organization, plus a taxonomy linking six game genres to design challenges.

    Source: §1 · p. 2, §2.1 · p. 2, §10 · p. 33

  • Results

    The review presents a coherent account of single-agent components, multi-agent coordination and organization, and genre-specific requirements ranging from low-latency action to open-ended sandbox goals.

    Source: §10 · p. 33

  • Takeaways & Limitations

    Games provide safe, controllable, cost-effective, and reproducible settings for examining interactive intelligence across varied levels of complexity.

    Source: §1 · pp. 1–2

    Current multi-agent simulations can rely on shared goals, cooperation-oriented prompts, or predefined role templates, constraining claims about emergent social behavior.

    Source: §9.4 · p. 31

Abstract

from arXiv · show

Game environments provide rich, controllable settings that stimulate many aspects of real-world complexity. As such, game agents offer a valuable testbed for exploring capabilities relevant to Artificial General Intelligence. Recently, the emergence of Large Language Models (LLMs) provides new opportunities to endow these agents with generalizable reasoning, memory, and adaptability in complex game environments. This survey offers an up-to-date review of LLM-based game agents (LLMGAs) through a unified reference architecture. At the single-agent level, we synthesize existing studies around three core components: memory, reasoning, and perception-action interfaces, which jointly characterize how language enables agents to perceive, think, and act. At the multi-agent level, we outline how communication protocols and organizational models support coordination, role differentiation, and large-scale social behaviors. To contextualize these designs, we introduce a challenge-centered taxonomy linking six major game genres to their dominant agent requirements, from low-latency control in action games to open-ended goal formation in sandbox worlds. A curated list of related papers is available at https://github.com/git-disl/awesome-LLM-game-agent-papers

1 Introduction

LLMs offer capabilities relevant to interactive agents, but their training on static text differs from intelligence grounded in sensorimotor interaction. This survey treats games as a rich, controllable testbed and reviews LLM-based game agents as a distinct research area.

1 Introduction

  • LLMs trained on static text differ from human intelligence grounded in repeated perception and action, where feedback helps improve behavior.Embodied cognition emphasizes situated environmental interaction rather than disembodied symbol manipulation.Source: §1 · p. 1
  • Games offer safe, controllable, cost-effective, and reproducible environments that approximate aspects of real-world interaction across varied complexity.They provide a richer, more embodied testbed than dialogue, web navigation, or API tool use.Source: §1 · pp. 1–2
  • Traditional game agents use predefined or learned state–action mappings, whereas LLM-based agents use language for goals, context, reasoning, reflection, and communication.Source: §1 · p. 2
  • Prior surveys either treat games as one application among others or focus on game development and content, leaving LLM-based game agents underexplored.The survey responds with a reference architecture for single- and multi-agent analysis and a taxonomy linking genres to design requirements.Source: §1 · p. 2

2 Research Methodology

The survey examines how LLM game agents are designed and how game genres relate to those designs. It uses a unified architecture and a structured search, screening, and snowballing process to review the literature.

2.1 Research Objectives and Questions

  • The survey’s objectives are to identify common LLMGA design patterns and relate game context to agent architecture through questions on components and genre requirements.It organizes single-agent components as memory, reasoning, and perception-action, and multi-agent design as communication and organization.Source: §2.1 · pp. 2–3
  • A challenge-centered taxonomy maps six representative game genres to agent-design demands, including role fidelity in role-playing games.The mapping is intended to structure prior work and offer practical guidance for future LLMGA development.Source: §2.1 · p. 3

2.2 Source Retrieval Process

  • The review searched four sources for publications from January 2022 to June 2026, used six-author title, abstract, and full-text screening, then backward snowballing.At least two independent include votes were required to retain a record after the initial screening.Source: §2.2 · p. 3

3 Overview

The overview frames LLM game agents through a single-agent perception-action loop, a multi-agent coordination framework, and a genre taxonomy that links game demands to design challenges.

3.1 LLM-based Game Agent (LLMGA) Framework

  • A central LLM connects memory, reasoning, and perception-action interfaces in a gameplay loop that turns evolving observations into actions.Working memory supports short-term processing while long-term memory accumulates experience; reasoning may use prompting or training, and interfaces translate observations and proposals between the agent and game.Source: §3.1 · pp. 3–4

3.2 Multi-LLMGA Framework

  • Multi-agent games require designed communication for constrained information sharing and organizational structures to manage decision authority, responsibilities, and coordination.Organizational structures can reduce coordination complexity and prevent uncontrolled interaction among agents.Source: §3.2 · pp. 4–5

3.3 Game Taxonomy for LLMGA Design

  • Game-agent design depends on genre because action games demand faster reactions than strategy games, while requiring less reasoning.Source: §3.3 · p. 5
  • The challenge-centered taxonomy links six gameplay genres to distinct design demands, including low-latency response, stateful world modeling, role fidelity, opponent-aware planning, and dynamics fidelity.The taxonomy merges narrower categories and adds sandbox games to align with existing LLMGA studies.Source: §3.3 · p. 5
  • Sandbox games make open-ended goal progression a design challenge because players set objectives, explore, and build.Agents may generate self-directed goals, decompose them hierarchically, and accumulate reusable skills for long-term play.Source: §3.3 · p. 6

4 Memory System of LLMGA

The survey separates memory into a limited, transient working memory and a durable long-term store. It reviews mechanisms for maintaining recent information and for consolidating, structuring, and retrieving durable memories.

4 Memory System of LLMGA

  • LLMGAs use working memory for recent processing and long-term memory for information retained beyond its limited span.The survey groups working-memory mechanisms as context extension, compression, and active maintenance, and long-term mechanisms as consolidation, structuring, and retrieval.Source: §4 · p. 6

4.1 Working Memory

  • Position-based context extension modifies positional encodings to enable length extrapolation, while other approaches segment or sparsely encode long inputs.Examples include frequency-aware YaRN, per-dimension rescaling in LongRoPE, coordinated segments in PCW, and sparse positional encoding in PoSE.Source: §4.1 · p. 7
  • Memory compression condenses lengthy inputs into compact representations, using virtual tokens or recursively summarized sequences to preserve salient content.Examples include segment-level summary vectors, reusable prompt tokens, chain-of-summarization, and NUGGET's structured summaries.Source: §4.1 · p. 7
  • In PokéLLMon, agents repeatedly switch Pokémon instead of attacking, and adopting chain-of-thought further exacerbates this short-term inconsistency.Last-Thoughts carries the prior reasoning trace forward, while belief-state, MEM1, and HiAgent approaches maintain compact state across steps.Source: §4.1 · p. 8, Figure 3 · p. 8

4.2 Long-Term Memory

  • Long-term memory consolidation decides what and when to commit from transient buffers, using signals such as event importance, goal relevance, task outcomes, or learned controllers.Generative Agents and MemoryBank use thresholds; Voyager commits successful code, while later approaches include policy-controlled or trainable updates.Source: §4.2 · pp. 8–9
  • Long-term memory structures range from observation chunks and key–value pairs to hierarchical trees, graphs, and knowledge encoded in model parameters.Examples include Place Event Memory, trajectory hierarchies, knowledge graphs, and fine-tuned parametric memory.Source: §4.2 · pp. 9–10
  • Retrieval reactivates stored information for current reasoning, using metadata, semantic similarity, exemplars, or graph and tree traversal.Retrieved content can include code, action plans, successful trajectories, or goal prerequisites; some systems also use implicit parametric knowledge.Source: §4.2 · p. 10

5 Reasoning of LLMGA

In LLM-based game agents, reasoning turns perceived and retrieved information into plans, decisions, and explanations. The survey groups reasoning approaches into instruction-guided prompting and fine-tuning, with methods that use intermediate thoughts, feedback, or training signals.

5 Reasoning of LLMGA

  • Reasoning transforms perceived and retrieved information into plans, decisions, and explanations, and the survey organizes its mechanisms into instruction-guided and fine-tuning approaches.The reasoning process unfolds through language-generated intermediate thoughts that guide actions.Source: §5 · pp. 10–11, Figure 5 · p. 11

5.1 Instruction-Guided Reasoning

  • Instruction-guided reasoning uses chain-of-thought for a single path, search to explore multiple paths, and reflection to iteratively improve reasoning with self-critique or external signals.These approaches are elicited by prompts at inference time.Source: §5 · p. 10, §5.1 · p. 11
  • Single-path reasoning can be inconsistent and propagate early errors, while search-based methods generate multiple candidate thoughts to select, aggregate, or revise.Self-Consistency uses majority voting across independently generated chains; Tree-of-Thoughts selects promising thoughts step by step.Source: §5.1 · p. 12
  • Reflective reasoning uses outcomes and feedback from prior actions to adjust future behavior, incorporating both failure signals and insights from successful trajectories.Signals can include explicit environment feedback, self-critique, heuristic feedback, or comparisons between successful and failed experiences.Source: §5.1 · pp. 12–13

5.2 Fine-tuning for Improving Reasoning

  • Fine-tuning improves reasoning and action generation through expert-trajectory imitation, reward-based reinforcement learning, or preference comparisons between better and worse trajectories.Supervised fine-tuning may use human, agent, or teacher-LLM demonstrations; rejection sampling filters candidate trajectories before training.Source: §5.2 · pp. 13–14
  • High-quality gameplay supervision lets LLMs approach strong game AIs across eight complex card games, while ChessLLM reaches 1788 Elo against Stockfish after training on complete chess trajectories.The survey also reports that SOTOPIA-π’s 7B model matches GPT-4 on social-goal completion.Source: §5.2 · p. 14
  • Reinforcement learning commonly optimizes policies with rewards, while reasoning-focused variants train on thought paths or use value-guided search; process reward models evaluate intermediate steps.The survey also describes self-play and learning-through-play as ways to supply learning signals without hand-specified rewards.Source: §5.2 · pp. 14–15
  • Preference optimization aligns agents using preferred versus dispreferred trajectories or reasoning steps, and can avoid an explicit reward model.ETO applies trajectory-level preferences to successful and failed rollouts, while IPR pairs reasoning steps using average Monte Carlo reward.Source: §5.2 · p. 15

6 Perception and Action Interfaces of LLMGA

Perception and action interfaces connect LLM-based game agents to their environments. The survey organizes perception by observation modality and action grounding by the form of executable operations.

6 Perception and Action Interfaces of LLMGA

  • Perception interfaces convert game states into representations an LLM can process, while action interfaces ground decisions as admissible in-game operations.The survey frames these interfaces as the input and output sides of the agent’s continuous perception-action loop.Source: §6 · p. 15, Figure 7 · p. 15

6.1 Perception Interface

  • Textual observations can be consumed directly when language is the environment’s primary medium, whereas symbolic observations summarize structured state variables but can omit important visual distinctions.Symbolic inputs may encode health, inventory, coordinates, or object properties.Source: §6.1 · pp. 15–16
  • Visual perception translates rendered frames into text descriptions or uses multimodal LLMs to interpret images directly, though direct visual models may need mechanisms to correct perception errors or uncertainty.Vision-to-text methods may use object detectors or pretrained encoders; direct approaches feed images or pixels to the model.Source: §6.1 · p. 17

6.2 Action Interface

  • High-level actions select discrete options or generate parser-constrained commands, with correction and constrained-decoding methods helping ensure admissibility.Length normalization can mitigate constrained decoding’s bias against valid but longer commands.Source: §6.2 · p. 17
  • Low-level controllers translate an LLM’s intent into timestep-level control signals, while code-based actions execute structured commands that can be checked and composed into reusable skills.Controllers may use heuristic planning or learned policies; code interfaces can detect invalid commands through parsing or execution feedback.Source: §6.2 · p. 18

7 Multi-LLMGA Framework

The multi-LLMGA framework treats coordination as message exchange under game-specific constraints and organizational design, covering communication, topology, division of labor, scalability, and robustness.

7.1 Communication Protocol

  • Game communication protocols operate under partial observability and limited bandwidth, specifying messages and how receivers integrate them into decisions.Messages may convey observations, inferred beliefs, or intended actions; summarizing observations can reduce redundant or low-value exchange.Source: §7.1 · p. 19
  • When messages conflict with internal models, agents reconcile them through partner-belief correction, debate and confidence-weighted voting, or probabilistic belief updates.Examples include ProAgent, ReConcile, ECON, BEACOF, and CSP4SDG.Source: §7.1 · p. 20

7.2 Organizational Structure

  • Organizational topology trades scalability, robustness, and latency; centralized, decentralized, hierarchical, and partitioned designs distribute authority and coordination differently.Centralized designs risk bottlenecks, decentralized ones can incur conflicts or redundant actions, and partitioning can weaken global consistency.Source: §7.2 · p. 20
  • Task allocation may be prefixed, dynamic, or emergent, trading reliable division of labor against adaptability and possible redundancy.Prefixed roles can be rigid as conditions change, while dynamic allocation can cause agents to converge on the same role.Source: §7.2 · p. 21
  • Thousands of Minecraft agents spontaneously specialize into professions, with social awareness reducing redundancy and memory reinforcing persistent roles.Agents shift tasks when many peers perform one, and repeated behavior helps stabilize identities such as farmer or miner.Source: §7.2 · p. 21
  • Sequential cognition limits early systems to dozens of agents; Project Sid’s PIANO parallelizes six modules, with a controller selecting among concurrent outputs [7].The controller transmits its selection to other modules to prevent incoherence between simultaneous outputs.Source: §7.2 · p. 21
  • Memory depth is reported as key to commonwealth emergence and reduced social disorder, yet LLM-augmented agents still struggle to cooperate effectively under social dilemmas.Artificial Leviathan links memory depth to the rise of a sovereign [40]; Melting Pot Commons Harvest evaluations report cooperation without effective collaboration.Source: §7.2 · p. 22

8 Gameplay Taxonomy for LLMGA Design

The taxonomy links six game genres to distinct agent-design challenges, from real-time control and persistent world modeling to role fidelity, goal formation, opponent-aware planning, and simulation fidelity.

8 Gameplay Taxonomy for LLMGA Design

  • The taxonomy maps representative game genres to design challenges, drawing on established classifications while excluding narrower genres and adding sandbox play.It frames genre as relevant to agent capabilities, from fast action-game cycles to long-horizon strategy planning.Source: §8 · p. 22

8.1 Action Games: Low-Latency Response

  • Action-game studies find that visual processing can worsen play through latency, while deeper reasoning can lower win rates; hybrid agents delegate reflexes to low-level policies.In Street Fighter, shallower reasoning can improve responsiveness and overall win rates; two-tier Overcooked systems reserve high-level reasoning for the LLM.Source: Figure 9 · p. 22, §8.1 · p. 23

8.2 Adventure Games: Stateful World Modeling

  • Adventure agents need stateful world models for long quests; memory, subgoal and precondition tracking, and adaptive models support coherent replanning, though sparse feedback and ambiguous goals remain difficult.NetPlay still struggles in NetHack despite tracking interaction history.Source: §8.2 · pp. 23–24

8.3 Role-Playing Games: Role Fidelity

  • Role fidelity requires sustained alignment between a character’s identity, dialogue, and actions; persistent role memory and role-play fine-tuning address drift beyond prompt-based role cards.RoleLLM separates private beliefs from public discourse records, while CharacterLLM, RoleLLM, and CoSER fine-tune on curated role-play data.Source: §8.3 · pp. 24–25

8.4 Sandbox Games: Open-Ended Goal Progression

  • Sandbox agents must generate their own objectives and execute them through hierarchical plans; plan correction, reusable skills, and learned world models support open-ended progression.Voyager builds a self-directed curriculum and skill library, while DEPS repairs plans using execution outcomes.Source: §8.4 · pp. 25–26, Figure 12 · p. 25

8.5 Strategy Games: Opponent-Aware Planning

  • Strategy games require multi-step planning around possible opponent intentions; explicit lookahead and opponent modeling support play, but models remain vulnerable to misdirection and rationality gaps.PokéLLMon agents can be baited by human misdirection, and analyses find persistent gaps from human rationality.Source: §8.5 · pp. 26–27

8.6 Simulation Games: Dynamics Fidelity

  • Simulation agents must produce behavior consistent with governing dynamics; cognitive and social mechanisms support plausible behavior, but studies caution that human-like socialization may not genuinely emerge.Validation has reproduced several human findings, while diagnostic studies retain concerns about emergent socialization.Source: §8.6 · pp. 27–28, Figure 14 · p. 28

9 Discussion and Open Challenges

The discussion synthesizes how architectural choices answer two questions: how LLMGAs are designed and how game genres shape their requirements, while identifying open challenges.

9.1 Memory System

  • Memory systems must link working and long-term stores, while future systems need coherent world models and selective consolidation rather than simply storing more.World-model memory is especially important in adventure games with long-horizon exploration and partial observability.Source: §9.1 · p. 29

9.2 Reasoning Mechanism

  • Reasoning approaches still rely on narrow feedback, while strategy games require opponent-aware planning across alternating moves or uncertain hidden states.The discussion proposes using textual, environmental, and self-critique signals as implicit reinforcement, beyond numeric rewards.Source: §9.2 · pp. 29–30

9.3 Perception-Action Interface

  • Perception and action interfaces trade representational richness and control precision against efficiency; action games need hierarchical strategies that meet strict latency constraints.The proposed direction separates slower strategic intents from lightweight, reactive low-level controllers.Source: §9.3 · p. 30

9.4 Multi-LLMGA System

  • Multi-agent designs use communication protocols and organizational structures to manage information exchange, decision flow, task allocation, and coordination across genres.Role-playing games emphasize selective disclosure, strategy games emphasize rapid intention sharing, and large simulations emphasize scalable organization.Source: §9.4 · pp. 30–31
  • Reported emergent roles and norms may reflect seeded goals, prompts, or role templates, while a lack of open, reproducible platforms limits systematic evaluation.The discussion calls for testing the conditions under which collective dynamics genuinely arise.Source: §9.4 · p. 31

9.5 Game Environments and Benchmarks

  • Many existing benchmarks use templated tasks, limited actions, and shallow dynamics; stronger environments should test compositional long horizons, consistent rules, breadth, and persistence.The proposed benchmarks should expose architectural weaknesses rather than reward pattern-matching templates.Source: §9.5 · p. 31
  • Evaluation should diagnose agent components beyond win and task-success rates, using game-specific measures and judgment-based protocols for role fidelity or behavioral coherence.The discussion describes qualitative ratings of consistency, plausibility, and coherence, with expert rubrics and human spot-checks as safeguards for LLM-based judgments.Source: §9.5 · pp. 31–33

10 Conclusion

The conclusion presents a unified analytical view of LLM-based game agents, spanning individual-agent components and multi-agent coordination. It also maps game genres to design requirements and identifies challenges for future research.

10 Conclusion

  • The survey organizes single-agent studies around memory, reasoning, and perception-action interfaces, describing how agents perceive, think, and act through language.It adds a multi-agent framework for communication protocols and organizational structures governing coordination, task allocation, and large-scale stability.Source: §10 · p. 33
  • Its challenge-centered taxonomy maps six major game genres to dominant agent requirements, from low-latency action-game response to open-ended goal generation in sandbox worlds..Source: §10 · p. 33
  • Together, these perspectives offer a coherent view of language-enabled agents in interactive games and outline key challenges for the field’s next research stage.Source: §10 · p. 33

Related papers

Loading 2404.02039v5…