PaperlayerView exact revision on arXiv

Source-linked AI summary

TowerMind: A Tower Defence Game Learning Environment and Benchmark for LLM as Agents

Dawei Wang, Chengming Zhou, Di Zhao, Xinyuan Liu, Marci Chi Ma, Gary Ushaw, Richard Davison

arXiv:2601.05899v2cs.AI

TL;DR

RTS games offer a way to evaluate LLM planning and decision-making, but existing environments either demand substantial computation or lack textual interfaces. TowerMind is a lightweight, multimodal tower-defense benchmark with hallucination evaluation; its experiments report a gap between LLMs and human experts and identify behavioral shortcomings.

  • Problem

    RTS benchmarks can be computationally demanding, while lightweight RTS environments lack textual observations and action interfaces needed for LLM evaluation.

    Source: §1 · pp. 1–2

  • Method

    TowerMind is a lightweight tower-defense environment with pixel, textual, and structured observations, customizable levels, and action executability metrics for hallucination evaluation.

    Source: §1 · p. 2, §3.2 · p. 4

  • Results

    On score, Claude 3.7 Sonnet in language-only and GPT-4.1 in vision-language still lag human experts by 62% and 58%, respectively.

    Source: §4.3 · p. 6

  • Takeaways & Limitations

    TowerMind offers a practical platform for LLM and reinforcement-learning research and may facilitate development of more capable AI agents.

    Source: §6 · p. 7

    LLMs often fail to validate long-term tower-placement plans and interpret available actions only at a surface level, lacking understanding of strategic use and context.

    Source: §4.4 · pp. 6–7

Abstract

from arXiv · show

Recent breakthroughs in Large Language Models (LLMs) have positioned them as a promising paradigm for agents, with long-term planning and decision-making emerging as core general-purpose capabilities for adapting to diverse scenarios and tasks. Real-time strategy (RTS) games serve as an ideal testbed for evaluating these two capabilities, as their inherent gameplay requires both macro-level strategic planning and micro-level tactical adaptation and action execution. Existing RTS game-based environments either suffer from relatively high computational demands or lack support for textual observations, which has constrained the use of RTS games for LLM evaluation. Motivated by this, we present TowerMind, a novel environment grounded in the tower defense (TD) subgenre of RTS games. TowerMind preserves the key evaluation strengths of RTS games for assessing LLMs, while featuring low computational demands and a multimodal observation space, including pixel-based, textual, and structured game-state representations. In addition, TowerMind supports the evaluation of model hallucination and provides a high degree of customizability. We design five benchmark levels to evaluate several widely used LLMs under different multimodal input settings. The results reveal a clear performance gap between LLMs and human experts across both capability and hallucination dimensions. The experiments further highlight key limitations in LLM behavior, such as inadequate planning validation, a lack of multifinality in decision-making, and inefficient action use. We also evaluate two classic reinforcement learning algorithms: Ape-X DQN and PPO. By offering a lightweight and multimodal design, TowerMind complements the existing RTS game-based environment landscape and introduces a new benchmark for the AI agent field. The source code is publicly available on GitHub(https://github.com/tb6147877/TowerMind).

1 Introduction

TowerMind addresses the lack of lightweight RTS environments with textual observations through a multimodal, customizable tower-defense environment for evaluating LLM planning and decision-making. Its evaluation reports a substantial gap between LLMs and human experts, alongside behavioral shortcomings in planning validation, multifinality, and action use.

Motivation

  • RTS games test strategic planning and real-time decisions, but SC2LE-based benchmarks are costly and lightweight alternatives lack textual observations and action interfaces.The paper presents TowerMind as addressing the lack of lightweight RTS environments with textual observation capabilities.Source: §1 · pp. 1–2

Contributions

  • TowerMind is a lightweight, multimodal tower-defense environment for evaluating LLM planning and decision-making, with hallucination analysis and customizable levels.Its metrics consider both in-game score and action executability; a graphical editor supports custom levels ranging from trivially easy to extremely difficult or structurally unique.Source: §1 · p. 2

Motivation

  • TowerMind uses tower defense to isolate evaluation of planning and decision-making from opponent unpredictability while retaining strategic gameplay.Fixed tower locations and predefined enemy roads also facilitate analysis of LLM strategies.Source: §1 · p. 2

Contributions

  • Commercial LLMs outperform open-source models, yet remain substantially behind human experts; evaluations also reveal planning-validation, multifinality, and action-use shortcomings.The paper also evaluates Ape-X DQN and PPO, finding TowerMind challenging and broadening the diversity of RL benchmarks.Source: §1 · p. 2

2 Related Work

Prior work evaluates LLM planning and decision-making in interactive environments and RTS benchmarks, while TowerMind is positioned as a computationally efficient alternative with multimodal observations. The paper also identifies a lack of dedicated, sufficiently challenging tower-defense environments for the research community.

Long-Term Planning and Decision-Making with LLMs

  • Interactive planning benchmarks cover operating systems, web tasks, and other environments, while TowerMind offers an RTS-style, more computationally efficient alternative.Examples include PlanGenLLMs [Wei et al. 2025], AGENTBENCH [Liu et al. 2023], and PLANET [Li et al. 2025].Source: §2 · p. 3

RTS Game-Based Environments and Benchmarks

  • SC2LE-based LLM RTS benchmarks face high computational demands, while TowerMind extends lightweight RTS environments with multimodal observations and agent-research functionalities.The comparison of lightweight environments is scoped to supported features in Table 1.Source: §2 · p. 3, Table 1 · p. 3

Tower Defense Games and AI Research

  • The related work describes tower defense as a rule-convergent RTS subgenre with simple mechanics and strategic depth, but notes a lack of dedicated, sufficiently challenging TD environments.The only TD game then available to the research community was a limited ELF module lacking tower and enemy variety and unit control.Source: §2 · p. 3

3 The TowerMind Environment

TowerMind is a Unity ML-Agents tower-defense environment where players defend a base across levels with distinct maps and enemy configurations. Its game mechanics, multimodal interface, difficulty model, and customization support planning and decision-making research.

3 The TowerMind Environment

  • The environment is built on Unity and extended into an AI environment with the Unity ML-Agents Toolkit.Source: §3 · p. 3

3.1 Game Mechanics

  • Each level sends sequential enemy waves toward the base along fixed roads, while tower points—including misleading points unable to engage enemies—shape defensive planning.Maps are square areas centered at (0.0, 0.0), with coordinates from −3.0 to 3.0.Source: §3.1 · p. 3
  • Towers provide relatively static strategic defense, while controllable knights and a stronger hero support tactical action; varied enemies and collectible gold prevent a single fixed clearing strategy.The three tower types specialize in single-target damage, area attacks, or summoning knights; fog of war adds partial observability and uncertainty.Source: §3.1 · p. 4

3.2 Environment Interface

  • Observations include a 512×512×3 pixel image, JSON text with semantic field names, and a flattened structured state representation of the same underlying information.The environment conforms to the OpenAI Gym standard.Source: §3.2 · p. 4
  • Actions combine continuous map coordinates with a discrete action type, and only commands valid for the current game state and rules are executable.Examples of invalid actions include building where no tower point exists or acting without sufficient gold; Figure 2 illustrates the map elements and 12 action types.Source: §3.2 · p. 4, Figure 2 · p. 5
  • Each level is a finite episode ending when the base reaches zero health or all waves are eliminated, with a sparse −1.0 reward for each enemy reaching the base.In the experiments, actions are executed every 16 game steps by default.Source: §3.2 · p. 4

3.3 Levels and Difficulty

  • Five built-in levels increase in difficulty, which is modeled as the sum of road, tower, enemy, and resource components.The described components use road and tower counts, enemy types and average enemies per wave, and level gold and tower sell-back settings.Source: §3.3 · pp. 4–5

3.4 Environment Customizability

  • Researchers can create levels with a graphical editor, modify nearly all game parameters, and selectively enable or disable features such as debugging and human-trajectory recording.The customization covers level, parameter, and feature settings.Source: §3.4 · p. 5

4 Evaluation for LLMs

The evaluation compares LLMs with human experts across five levels and two input modalities, finding substantial performance gaps and recurring weaknesses in planning and action use.

4.1 Evaluation Setting

  • Five human experts and several commercial and open-source LLMs were evaluated on five levels using score and valid action rate under language-only and vision-language settings.Each model was tested with five random seeds per level and modality; vision-language input added a 512 × 512 × 3 pixel observation to the prompt.Source: §4.1 · p. 5, §4.2 · p. 5

4.3 Quantitative Analysis

  • Claude 3.7 Sonnet and GPT-4.1 still lag human experts by 62% and 58%, respectively, in their best-performing modalities.Claude 3.7 Sonnet led in language-only score, while GPT-4.1 led in vision-language score; all models were at least 84% below experts on Level 5.Source: §4.3 · p. 6
  • Most evaluated models scored better with vision-language input, while hallucination increased with level difficulty and was especially problematic for several open-source models.Llama 3.2 (90B/11B) did not show the score improvement; Qwen 2.5-VL 7B and Llama 3.2 11B fell below the random baseline on some levels for valid action rate.Source: §4.3 · p. 6, §4.2 · p. 6

4.4 Qualitative Analysis

  • Trajectory analysis found that LLMs built useless towers despite available evidence, did not combine goals in single actions, and often misused available actions.Examples included failing to upgrade towers despite sufficient gold, sending knights to empty areas, and using the hero’s area-of-effect skill without enemies.Source: §4.4 · pp. 6–7

4.5 Insights and Future Directions

  • The authors argue that visual input can provide information beyond text, while evaluation should distinguish rule-consistent actions from actions that effectively achieve goals.They also call for outcome-driven validation that assesses the effects or implications of generated content.Source: §4.5 · p. 7

5 RL Benchmark

The RL benchmark trains Ape-X DQN and PPO on five levels and evaluates them using pixel-based and structured game-state observations. After 100 million environment steps, both algorithms solve simpler levels to some extent but remain substantially below human experts.

5 RL Benchmark

  • Each algorithm was trained three times with different random seeds for 100 million environment steps and evaluated using five random seeds.Evaluation used the same score metric as the LLM-based evaluation.Source: §5 · p. 7
  • Figure 3 reports benchmark-level scores normalized relative to human experts with standard-error bars, while Figure 4 shows training curves over sample counts in millions with 95% confidence intervals.Source: Figure 3 · p. 7, Figure 4 · p. 7
  • After 100 million environment steps, Ape-X DQN and PPO solve simpler levels to some extent but remain substantially below human experts, indicating TowerMind is challenging for RL.The two algorithms were trained on five benchmark levels using pixel-based and structured game-state observations.Source: §5 · p. 7

6 Conclusion

The conclusion presents TowerMind as a lightweight, multimodal tower-defense environment for evaluating LLMs. Its evaluations reveal a substantial LLM–human performance gap, and the authors describe the environment as a practical platform for AI-agent research.

6 Conclusion

  • TowerMind is a lightweight tower-defense environment with multimodal observations designed to evaluate LLMs.Source: §6 · p. 7
  • Evaluations reveal a substantial gap between current LLMs and human experts in long-term planning and decision-making, alongside roles for visual input, hallucination, and misleading information.Source: §6 · p. 7
  • The authors state that TowerMind can also be used for reinforcement-learning research and may serve as a practical platform for developing more capable AI agents.Source: §6 · p. 7

A Game Mechanics Additional Details

These additional mechanics describe how actions map to tower points, how units and enemies differ, and how resources and visibility shape play. User-defined seeds make outcomes reproducible for a fixed action sequence.

A Game Mechanics Additional Details

B Environment Interface Additional Details

TowerMind exposes game state through pixel, textual, and structured observations, and records action validity and errors. Tower operations use bounded coordinate regions, while invalid actions are ignored.

B Environment Interface Additional Details

  • The three observation modalities are a 512 × 512 RGB game image, JSON-formatted textual state, and a 759-value flattened structured state.Source: B Environment Interface Additional… · p. 13
  • Actions are valid only when permitted by game rules and current state; invalid actions are ignored, with validity affected by coordinates, available gold, and whether the hero is alive.Source: B Environment Interface Additional… · p. 13
  • The Agent Last Action Info field records action coordinates, type, execution status, and any error code, with Listing 7 mapping codes to invalid-action reasons.Source: B Environment Interface Additional… · p. 13
  • Tower construction or related operations require coordinates within one of the predefined 0.5 × 0.5 bounding boxes; otherwise the action is invalid and not executed.Source: Figure 7 · p. 14

C Additional Details of the Benchmark Levels

The benchmark levels’ difficulty is calculated from road, tower, enemy, and resource components, using design-time maxima or minima as normalization bounds.

C Additional Details of the Benchmark Levels

D Additional Details of Environment Customizability

This section details TowerMind’s environment customizability, including map and configuration workflows and zero-shot prompts that combine game instructions, action and configuration information, interaction history, structured game state, and image observations to elicit constrained JSON actions.

D Additional Details of Environment Customizability

  • A GUI level editor lets users draw road shapes, place waypoints and tower points by drag and drop, then export maps as images with matching JSON files for import.Figure 8 illustrates the map-editing workspace and placement interface.Source: D Additional Details of Environment… · p. 15, Figure 8 · p. 15
  • Environment configuration files are editable JSON, and EnvConfig controls auxiliary features such as action-space discretization preview and human gameplay recording.The files are located in StreamingAssets/Config, though the exact path may vary by platform.Source: D Additional Details of Environment… · p. 15

E Zero-shot Prompt

  • The zero-shot prompt combines the game objective and rules, natural-language action-space guidance, JSON configuration tables, and a default three-step observation–action history.The configuration tables cover towers, knights, the hero, reinforcements, and enemies.Source: E Zero-shot Prompt · p. 15
  • The prompt describes coordinate-based actions and their validity constraints, including costs, tower-point bounds, reinforcement cooldowns, and the hero’s movement rather than teleportation.It also outlines wave-based survival, health loss when enemies reach their destination, and Fog of War effects on units and attacks.Source: E Zero-shot Prompt · p. 16
  • Example state fields include hero status, revive countdown, death state, position, and current health.The excerpt shows a revive countdown of 10 and current health of 1600.Source: E Zero-shot Prompt · pp. 18–19
  • The structured state also includes real-time tower status and tower position, name, and knight-assembly position fields.The excerpt shows two Archer Tower entries.Source: E Zero-shot Prompt · p. 19
  • Additional state fields cover enemies, knights, dropped gold coins, and information about the agent’s last action.The excerpts also show a dropped coin position and remaining lifetime.Source: E Zero-shot Prompt · p. 19
  • The prompt requests an action as JSON containing floating-point X and Y coordinates and an integer action index, with no other text.An image observation is also provided.Source: E Zero-shot Prompt · p. 19

F Human Expert Performance

Five tower-defense-experienced university participants received a game guide and non-benchmark training levels before evaluation on the five benchmark levels. The study reports their evaluation setup and metrics, but the supplied passages do not include their results.

F Human Expert Performance

  • Five participants with prior tower-defense experience were recruited from a university environment and agreed to approximately 15 hours of study activities within one week.Source: F Human Expert Performance · p. 19
  • After 13 hours of training, each participant evaluated five benchmark levels with a unique random seed per level, using score and valid action rate as metrics.Their evaluation APM ranged from 40 to 110.Source: F Human Expert Performance · p. 19
  • Participants received a TowerMind game guide and five non-benchmark training levels, and were compensated £17.89 per hour.Training took place on participants’ personal laptops with a mouse and keyboard; the environment ran smoothly on each device.Source: F Human Expert Performance · p. 19

G Additional Experimental Results on LLM Evaluation

LLM evaluation used a single laptop, with episode times differing by input modality; reported values are normalized relative to the human expert baseline. Supplementary material provides correlation analysis between score and valid action rate.

G Additional Experimental Results on LLM Evaluation

  • LLM evaluation ran on one laptop; average episode time was approximately 7 minutes for language-only input and 15 minutes for vision-language input.The laptop had a 13th Gen Intel Core i7-13650HX CPU, 64 GB RAM, and an NVIDIA GeForce RTX 4060 Laptop GPU with 8 GB VRAM.Source: G Additional Experimental Results on… · p. 19
  • Evaluation values are normalized relative to the human expert baseline, using the human baseline and minimum possible score or valid action rate.The normalized measure is described for an LLM’s level score or valid action rate.Source: G Additional Experimental Results on… · pp. 19–20
  • Supplementary material reports correlation analysis between score and valid action rate.Source: G Additional Experimental Results on… · p. 20

H Additional Details on RL Experiments

The Ape-X DQN and PPO experiments used pixel-based and structured game-state observations, with training-oriented preprocessing of the environment. Training randomized benchmark levels across episodes.

H Additional Details on RL Experiments

  • Ape-X DQN and PPO used RLlib, pixel-based CNNs with approximately 3.1 million parameters, and were also evaluated with structured game-state observations.Training ran on a PC with an Intel i7-14700K CPU, NVIDIA RTX 3090 GPU, and 192 GB RAM.Source: H Additional Details on RL Experiments · p. 20
  • Training downsampled pixel observations from 512 × 512 × 3 to 128 × 128 × 3 and discretized coordinates into a 10 × 10 × 12 action space.The first two action variables were divided into 10 intervals from −3.0 to 3.0; action type remained unchanged, and a −5×10−4 step penalty was added.Source: H Additional Details on RL Experiments · p. 20, Figure 12 · p. 20
  • Each training episode began with a randomly selected benchmark level from the nine-level set.Source: H Additional Details on RL Experiments · p. 20
  • The CNN maps 128 × 128 × 3 pixel input to a 1200-dimensional policy output, while the MLP maps 759 structured inputs to the same output size.Source: Figure 10 · p. 20, Figure 11 · p. 20
Loading 2601.05899v2…