PaperlayerView exact revision on arXiv

Source-linked AI summary

Word2World: Generating Stories and Worlds through Large Language Models

Muhammad U. Nasir, Steven James, Julian Togelius

arXiv:2405.06686v1cs.CLcs.AI

TL;DR

Directly generating playable game levels from text remains challenging, particularly when narratives, characters, maps, and objectives must be coordinated. Word2World addresses this with a staged LLM pipeline that extracts story information, constructs worlds, and uses evaluation feedback. It generates playable levels 90% of the time and produces worlds described as highly coherent with their stories, while remaining limited in scope, expressiveness, mechanics, and deployment cost.

Abstract

from arXiv · show

Large Language Models (LLMs) have proven their worth across a diverse spectrum of disciplines. LLMs have shown great potential in Procedural Content Generation (PCG) as well, but directly generating a level through a pre-trained LLM is still challenging. This work introduces Word2World, a system that enables LLMs to procedurally design playable games through stories, without any task-specific fine-tuning. Word2World leverages the abilities of LLMs to create diverse content and extract information. Combining these abilities, LLMs can create a story for the game, design narrative, and place tiles in appropriate places to create coherent worlds and playable games. We test Word2World with different LLMs and perform a thorough ablation study to validate each step. We open-source the code at https://github.com/umair-nasir14/Word2World.

Introduction

Generating coherent game content across narratives, characters, and playable maps remains difficult, especially for zero-shot LLMs generating levels directly from text. Word2World addresses this by converting stories into narrative-driven, playable 2D worlds through staged information extraction and level design.

Introduction

  • Jointly generating narratives, characters, and playable world maps across games remains an open content-generation challenge.Existing approaches are typically narrow in the content types or environments they support.Source: Introduction · p. 1
  • Directly generating coherent 2D game levels from text is difficult because design must coordinate narratives, characters, pathways, environments, and objectives.These requirements must support both gameplay and narrative and visual coherence.Source: Introduction · pp. 1–2
  • The broader goal is to generate diverse, aesthetically pleasing content that remains coherent with its underlying story.Such a system could support game-design tooling, player-adaptive games, and user-generated content.Source: Introduction · pp. 1–2
  • Word2World converts stories into narratives and playable 2D game levels without task-specific fine-tuning.The system extracts character, tile, and goal information and integrates it into game-level design.Source: Introduction · p. 2

Word2World

Word2World uses staged LLM-based extraction and generation rather than direct prompting: it creates a story, derives world constraints, builds the map, and iteratively evaluates it. Tile retrieval and an LLM agent support concrete world construction and playability assessment.

Word2World

  • Word2World generates playable games from stories through multiple stages because direct prompt-based generation is unlikely to produce the desired result.The stages are presented in Figure 1 and Algorithm 1.Source: Word2World · p. 2

Procedural Content Generation

  • The pipeline extracts characters, tilesets, goals, important tiles, walkable tiles, and interactive tiles before generating the world in two passes.The first pass lays down the environment, while the second places characters and important interactive tiles; algorithmic fixes then normalize the map.Source: Procedural Content Generation · pp. 2–3
  • Each round evaluates the generated world with an LLM and conventional PCG checks, then feeds the previous world and scores back into later generation rounds.This feedback loop is intended to improve subsequent worlds.Source: Procedural Content Generation · p. 2

Tile Retrieval

LLM Agent

  • The LLM agent plans action sequences toward successive objectives using the world, positions, prior actions, and rewards as inputs.Available actions include movement, object pickup, and enemy attacks; rewards prioritize objective completion and proximity.Source: LLM Agent · p. 3, Figure 3 · p. 3

Evaluations

Word2World evaluates story-world coherence and game functionality through complementary LLM-based judgments and conventional PCG checks. The evaluation suite measures content accuracy, agent performance, playability, path efficiency, novelty, and completion.

Evaluations

  • The evaluation framework measures whether generated worlds are coherent with their stories, diverse, and playable.It combines LLM-based evaluations with conventional PCG checks.Source: Evaluations · p. 3

LLM-based Evaluations

  • LLM-based evaluation scores coherence between story and world, while LLM agent rewards indicate whether the world is solvable.Coherence considers the environment, character similarity, and protagonist objectives.Source: Evaluations · p. 3

Conventional PCG Checks

  • Conventional PCG checks use AStar agents to test playability and shortest path length across all objectives.Playability requires finding paths that complete the objectives.Source: Evaluations · p. 4
  • Novelty is measured by Euclidean distance from worlds generated in previous rounds, with a threshold determining whether a world is novel.Novel-and-playable worlds are checked for novelty before playability.Source: Evaluations · p. 4

Evaluation Metrics

  • Completion is true when the entire Word2World run finishes within 10 tries across the pipeline.Character and important-tile accuracy compare placed tiles against those specified in the story.Source: Evaluations · p. 4

Experiments

Experiments evaluate multiple LLMs and ablated Word2World variants across gameplay, novelty, coherence, rewards, completion, and tile-related measures. The study uses repeated runs and compares the full pipeline with versions that remove intermediate extraction or generation steps.

Experimental setup

  • Experiments use 10 runs, with 4–5 story paragraphs, 8 objectives, and a maximum of 15 important tiles.The authors selected these parameters after experimenting across different ranges.Source: Experiments · p. 4

Model comparison

  • Six LLMs are compared across Playability, Novelty, Novel and Playable, Coherence, LLM Agent Rewards, and Completion.Experiments use 10 runs; all metrics are counts except normalized LLM Agent Rewards collected after two episodes.Source: Experiments · p. 4

Ablation study

  • The ablation study tests whether Word2World’s intermediate steps are necessary by comparing direct, reduced-extraction, and simplified-generation variants.Variants remove goals, important-tile extraction, staged generation, or repeated rounds.Source: Experiments · pp. 4–5
  • Figure 5 compares Word2World and five ablations on Coherence, Playability, character-tile accuracy, and important-tile accuracy.The displayed variants are Word2World, one-round-generation, one-step-generation, direct-generation, no-goals-extracted, and No-important-tiles-extracted.Source: Experiments · pp. 4–5, Figure 5 · p. 5

Results

Word2World generates diverse, story-coherent playable worlds through staged information extraction, iterative generation, and evaluation. Results show that the full pipeline improves playability and coherence while supporting varied world sizes, agent traversal, and strong performance from larger language models.

Ablation Study Results

  • Word2World generates the most coherent worlds, while direct-generation performs poorly because it receives only the story without the extracted contextual information.One-round-generation and one-step-generation remain coherent when the extracted information is retained.Source: Ablation Study Results · p. 6
  • 90% of Word2World generations are playable, compared with 60% for one-round-generation, showing that iterative rounds and separate environment-character placement matter.The ablations also indicate that extracting goals is important for maintaining playability.Source: Ablation Study Results · p. 6

Correlation Between Input And Output

  • Increasing story paragraphs, objectives, and important tiles generally increases path length and supports more complex environments, but does not ensure coherence or character-tile accuracy.Coherence and character-tile accuracy are the main metrics not strongly correlated with these world-generation parameters.Source: Other Results · pp. 6–7, Figure 6 · p. 6

Diverse Worlds

  • Word2World produces worlds with varying map sizes and diverse environment-tile patterns, with a wider width distribution than height.The reported distribution supports variation in generated world dimensions and layouts.Source: Other Results · p. 7, Figure 7 · p. 7

LLM Agent Rewards

  • LLM agent rewards increase across episodes on worlds with 4, 6, and 8 objectives as prior action sequences and rewards provide feedback.The evaluation uses five episodes across ten seeds, with normalized rewards and shaded standard deviations.Source: Other Results · p. 7, Figure 8 · p. 7

LLM Comparisons

  • Claude-3-opus-20240229 leads Novelty, Coherence, and LLM Agent Rewards, while gpt-4-turbo-2024-04-09 leads Playability, Novel and Playable, and Completion.The two largest models perform almost similarly overall and substantially better than their smaller versions.Source: Other Results · p. 7, Table 1 · p. 7

Related work

Word2World builds on prior work in story-based generation, procedural content generation, game mastering, and computational game creativity. It differs by using an LLM pipeline to generate diverse components of playable worlds from stories.

Story-based game generation

  • Word2World differs from earlier story-to-world systems by generating all game components, supporting diversity across the resulting worlds.Earlier methods used knowledge graphs, trained neural networks, or search-based optimization to place objects from stories.Source: Related work · p. 8

Procedural Content Generation

  • Prior procedural content generation methods have used deep learning, evolutionary algorithms, and sequence models to generate levels for particular games.Examples include LSTM-based level generation and supervised LLM generation of Sokoban and Super Mario Bros. levels.Source: Related work · p. 8

Game Master systems

  • Word2World is framed as a Game Master that creates a plot and characters, then places story components in a top-down role-playing game.Related systems include AI Dungeon, CALYPSO, and LLMs trained on games to assist or act as Game Masters.Source: Related work · p. 8

Computational Game Creativity

  • The system belongs to multi-faceted game generation because its story, character, tile, goal, and world-generation stages each require computational creativity.This connects Word2World to earlier systems such as ANGELINA, which used evolutionary algorithms to create puzzle and platform games.Source: Related work · p. 8

LLM-based evaluation

  • Word2World also uses an LLM agent to evaluate world traversal, building on prior demonstrations of language-model agents playing games.The paper cautions that such agents require finely tuned prompts and that not all LLMs can play games.Source: Related work · p. 8

Discussion

The paper identifies scope, expressiveness, mechanics, cost, hardware, and narrative limitations for Word2World. Its current design targets top-down games and depends on retrieved tiles and the underlying LLM.

Scope and expressiveness

  • Word2World is limited to top-down games, predefined mechanics, and the expressive capability of its underlying LLM.These constraints restrict the range of game settings and behaviors the system can generate.Source: §1 · p. 8
  • World expressiveness is constrained by the tile dataset and unresolved scaling issues, such as placing a house across multiple tiles.The paper links these issues to limitations in world aesthetics and expressiveness.Source: §1 · p. 8

Mechanics and narrative

  • Word2World has no animation mechanism, and generated stories often reuse nearly the same events in different environments.These limitations affect both world presentation and narrative variety.Source: §1 · p. 8

Cost and deployment

  • Each generation costs approximately $0.5 to $1, and the system cannot run fully locally because it does not use an open-sourced LLM.The reported cost depends on the selected LLM and the number of generated tokens.Source: §1 · p. 8

Conclusion

Word2World is an LLM-based pipeline that generates playable worlds from stories while controlling content generation. The paper reports 90% playability, story-coherent worlds, and diverse environments with potential uses in industry and reinforcement learning.

Conclusion

  • 90% of Word2World generations produce playable levels, while the generated worlds are reported to remain highly coherent with their stories.The conclusion attributes this controlled generation to combining LLM content generation with information extraction.Source: Conclusion · p. 8
  • Word2World generates playable worlds from stories by leveraging LLMs to create diverse content and extract information needed for world construction.The system is presented as a novel LLM-based pipeline for story-driven game generation.Source: Conclusion · p. 8

Implications

  • The authors describe the direction as important for both the research community and industry.This statement accompanies the proposed uses of Word2World for drafting levels and generating reinforcement-learning environments.Source: Conclusion · pp. 8–9
  • Figure 9 illustrates examples of diversity in the generated worlds.The figure provides visual examples rather than a quantitative comparison.Source: Figure 9 · p. 9, Conclusion · p. 9
  • The generated environments are intended to support industry level drafting and reinforcement-learning paradigms that require diverse environments.The paper specifically mentions open-ended learning as a setting that needs an environment generator.Source: Conclusion · p. 9

Related papers

Loading 2405.06686v1…