PaperlayerView exact revision on arXiv

Source-linked AI summary

Can Language Models Serve as Text-Based World Simulators?

Ruoyao Wang, Graham Todd, Ziang Xiao, Xingdi Yuan, Marc-Alexandre Côté, Peter Clark, Peter Jansen

arXiv:2406.06485v1cs.CLcs.AI

TL;DR

Because building virtual-environment simulations can require extensive expert work, the paper asks whether LLMs can directly simulate text-based worlds. It defines and evaluates a state-transition prediction task, finding GPT-4 unreliable on non-trivial transitions and concluding that LLMs are not yet reliable text-world simulators.

  • Problem

    Virtual-environment simulations can require extensive work by human experts, motivating the question of whether LLMs can correctly predict how actions change world states.

    Source: §1 · p. 1

  • Method

    The paper defines LLM-Sim as predicting the next state, reward, and completion status from context, state, and action, and evaluates GPT-4 on text-based environments.

    Source: §2.1 · p. 2, §2 · p. 2

  • Results

    GPT-4 accuracy does not exceed 59.9% on transitions involving non-trivial state changes, and it is harder to predict environment-driven transitions than action-driven ones.

    Source: §1 · p. 2, §4 · p. 4, §5 · p. 5

  • Takeaways & Limitations

    The authors conclude that LLMs are not yet reliable text-world simulators without further innovation.

    Source: §1 · p. 2, §5 · p. 5

    The evaluated state spaces focus on common-sense and elementary scientific reasoning, not highly domain-specific areas such as physical or medical simulation.

    Source: §6.1 · p. 6

Abstract

from arXiv · show

Virtual environments play a key role in benchmarking advances in complex planning and decision-making tasks but are expensive and complicated to build by hand. Can current language models themselves serve as world simulators, correctly predicting how actions change different world states, thus bypassing the need for extensive manual coding? Our goal is to answer this question in the context of text-based simulators. Our approach is to build and use a new benchmark, called ByteSized32-State-Prediction, containing a dataset of text game state transitions and accompanying game tasks. We use this to directly quantify, for the first time, how well LLMs can serve as text-based world simulators. We test GPT-4 on this dataset and find that, despite its impressive performance, it is still an unreliable world simulator without further innovations. This work thus contributes both new insights into current LLM's capabilities and weaknesses, as well as a novel benchmark to track future progress as new models appear.

1 Introduction and Related Work

The paper asks whether language models can simulate text-based worlds, where building simulations otherwise requires extensive expert effort. It directly evaluates this ability and finds GPT-4 remains unreliable on non-trivial state transitions.

1 Introduction and Related Work

  • Simulation can require weeks or months of extensive work by human experts, motivating the question of whether LLMs can serve as an alternative.Source: §1 · p. 1
  • Text-based games express environments and their dynamics in natural language and have been used to study decision making, information extraction, and artificial reasoning [Côté et al., 2018; Fan et al., 2020; Urbanek et al., 2019; Shridhar et al., 2020; Hausknecht et al., 2020; Jansen, 2022; Wang et al., 2023; Ammanabrolu and Hausknecht, 2020; Adhikari et al., 2020; Wang et al., 2022].Source: §1 · p. 1
  • The study provides the first quantitative analysis of LLMs directly simulating virtual environments, using a structured JSON schema as a scaffold and probe across conditions.The work distinguishes direct simulation from approaches that generate symbolic code for formal planning or inference [Liu et al., 2023; Nottingham et al., 2023; Wong et al., 2023; Tang et al., 2024].Source: §1 · pp. 1–2
  • GPT-4 accuracy does not exceed 59.9% on transitions involving non-trivial world-state changes, and the authors conclude it is not yet a reliable simulator.The reported failures include transitions not directly related to agent actions and those requiring arithmetic, common-sense, or scientific reasoning.Source: §1 · p. 2

2 Methodology

The methodology formalizes text-game simulation as predicting state transitions and game progress from a context, state, and action. It introduces a transition dataset and evaluates single-step predictions under multiple task and context conditions.

2 Methodology

  • Text-game environments are represented as goal-conditioned POMDPs with states, actions, transitions, observations, rewards, context messages, and completion indicators.Source: §2 · p. 2

2.1 LLM-Sim Task

  • LLM-Sim asks a model to map context, current state, and action to the next state, reward, and game-completion status.The simulator is decomposed into action-driven transitions, environment-driven transitions, and game-progress prediction; models can output either the full state or a state difference.Source: §2.1 · p. 2

2.2 Data

  • BYTESIZED32-State-Prediction contains 76,369 transitions from 31 text games, with transition data collected from modified BYTESIZED32 games and trajectories near gold-label paths.Each game context can use human-written rules, LLM-generated rules, or no rules.Source: §2.2 · p. 3

2.3 Evaluation

  • Evaluation measures prediction accuracy for object properties and game progress, giving the model the ground-truth previous state and task context for each single-step prediction.The evaluation separates object properties from reward, termination, and goal achievement.Source: §2.3 · p. 3

3 Experiments

The experiments evaluate GPT-4 with in-context learning on full-state and state-difference predictions. The test sample is balanced across action verbs and static versus dynamic action-driven transitions.

3 Experiments

  • The experiments use in-context learning to evaluate GPT-4 in both full-state and state-difference prediction regimes.The model receives the previous state as JSON, the previous action, and a context message, then produces the subsequent state or its difference.Source: §3 · pp. 3–4
  • The test set contains 2,954 transition tuples, sampled to balance dynamic and static transitions for each action verb where possible.Dynamic transitions change the output state; static transitions leave it unchanged, and the sampling takes up to 10 of each per verb.Source: §3 · p. 4

4 Results

GPT-4's state-transition performance varies with transition type, state dynamics, and rule context; additional analyses identify error patterns and compare human accuracy.

  • 77.1% of dynamic action-driven transitions were simulated correctly at best, compared with at most 49.7% of dynamic environment-driven transitions.The results indicate that modeling underlying environmental dynamics is likely the most challenging part of LLM-Sim.Source: §4 · p. 4
  • Static transitions are substantially easier than dynamic transitions across most conditions.Dynamic transitions require matching the game engine's dynamics using the context, beyond determining whether a state changes.Source: §4 · p. 4
  • State-difference prediction improves static-transition performance by more than 10% but decreases performance on dynamic transitions.The authors suggest this may be because GPT-4 usually gets the response format right, while difference prediction adds output-format complexity.Source: §4 · p. 4
  • Providing game rules improves GPT-4 performance on all three simulation tasks in most conditions, with no obvious difference between human- and LLM-generated rules.Source: §4 · p. 4
  • 92.1% of test cases had correctly predicted game progress with rules in context, versus 61.5% without them.Source: §4 · p. 4
  • Human annotators reached 80% accuracy on the sampled Fact task, compared with 50% for GPT-4 on the sampled transitions.Variation among the four annotators was small.Source: §4 · p. 5
  • Errors concentrate on properties requiring arithmetic, common-sense, or scientific reasoning; combined predictions also leave some environment-driven properties unchanged.GPT-4 handles most simple Boolean properties well; the property-wise figure breaks performance down across whole, action-driven, and environment-driven transitions.Source: §4 · p. 5, Figure 2 · p. 5

5 Conclusion

The paper introduces a 76,369-transition benchmark for testing LLMs as text-world simulators and finds that GPT-4 is not yet reliable for this role. Its analysis highlights weak performance on environmental dynamics and reasoning-intensive transitions, with errors compounding across steps.

  • Across models and conditions, the best performance on state transitions involving non-trivial changes was 59.9%, leading the authors to conclude LLMs are not yet reliable text-world simulators.Source: §5 · p. 5
  • Further error analysis finds that LLMs handle user-action results better than environment-driven transitions and transitions requiring arithmetic, common sense, or scientific knowledge.Source: §5 · p. 5
  • After 10 steps, the authors estimate that average simulation accuracy would fall below 1% because errors accumulate across steps.Source: §5 · p. 5
  • BYTESIZED32-State-Prediction provides 76,369 virtual text-environment state transitions for testing LLMs as simulators.Source: §5 · p. 5

6 Limitations and Ethical Concerns

The evaluation is limited in model coverage, representation, and domain scope; the paper also cautions that LLM-generated misinformation or hallucinations may create risks in direct human-facing use.

6.1 Limitations

  • The study evaluates GPT-3.5 and GPT-4 but does not exhaustively test available language models, so other models may perform better.The authors position the work as a benchmark for existing and future models.Source: §6.1 · p. 6
  • The evaluation uses two JSON state-space representations, though other representational formats might perform better at simulation.The formats were chosen for popularity, compatibility with LLM pretraining formats, and comparison with gold-standard simulator output.Source: §6.1 · p. 6
  • State spaces cover common-sense and elementary scientific reasoning, not highly domain-specific areas such as physical or medical simulation.The authors describe this work as a stepping-stone toward simulators for high-impact domains.Source: §6.1 · p. 6

6.2 Ethical Concerns

  • The LLM-Sim task may be affected by misinformation or hallucinations introduced by the selected model.Source: §6.2 · p. 6
  • The authors warn against direct human-facing deployment, especially in educational settings with children, because simulators may generate misleading or dangerous suggestions.They give burning a house to boil water as an example and do not evaluate the ethical implications of such suggestions.Source: §6.2 · p. 6

A Model details

The experiments use specified GPT-3.5 and GPT-4 API models with deterministic JSON responses, at an approximate API cost of $5,000.

A Model details

B Game transition examples

The transition examples use the BYTE-SIZED32 wash-clothes game, selected to include both action-driven transitions and underlying game dynamics.

B Game transition examples

  • The wash-clothes game was chosen because it contains both action-driven transitions and game dynamics.Source: B Game transition examples · p. 8
  • Tasks predicting action transitions, environment-driven transitions, or game progress receive one in-context example; predicting everything receives two.Source: B Game transition examples · p. 8
  • For the two-example task, the examples are manually selected to show one action-driven state change and one change driven by game dynamics.Source: B Game transition examples · p. 8

C.1 LLM generated rules

The appendix describes how game rules are generated or written and how prompts ask models to predict action-driven, environment-driven, and game-progress changes. It also provides examples of the rule and state formats used.

C.1 LLM generated rules

  • GPT-4 generates object rules from object-class code, action rules from whole-game code, and score rules from game code; generated rules are manually checked for misinformation and offensive content.Object-rule prompts request critical properties and tick behavior; action-rule prompts ask for actions, constraints, and state changes, while score-rule prompts request win/loss conditions and rewards.Source: §C.1 · p. 8

C.2 Human-Written Action Rules

  • Human-written action rules specify each action's purpose and unordered constraints or conditional effects, with gameplay checks used to verify their accuracy.The expert annotator consults game descriptions and source code, including the corresponding action functions.Source: §C.2 · pp. 8–9

C.3 Human-Written Object Rules

  • Human-written object rules describe object purpose, inheritance, properties and defaults, plus tick-driven changes when an object has a tick function.Annotators consult game descriptions and source code and check the rules against gameplay.Source: §C.3 · p. 9

C.4 Human-Written Score Rules; D.3 Prompt Example: FR (Game Progress)

  • Score rules specify win/loss conditions and reward earning; the human-written version is created by a game author using the description and score-function code.The game-progress prompt asks the model to predict score, game-over status, and win status, using score rules and the prior state as context.Source: §C.4 · p. 9, §D.3 · p. 10

D.1.1 Full State Prediction; D.1.2 State Difference Prediction

  • Full-state prompts request the new state in the input JSON format, while state-difference prompts return added or changed objects and removed-object UUIDs.Prompts can include human-written rules, LLM-generated rules, or no rules.Source: §D.1.1 · p. 9, §D.1.2 · p. 9, D Prompts · p. 9

D.2 Prompt Example: Fenv; D.2.2 State Difference Prediction

  • Environment-driven prompts ask for the next state without considering agent actions, using object tick functions to describe those changes.The same full-state or difference-output distinction is used for these prompts.Source: §D.2.1 · p. 10, §D.2.2 · p. 10

D.4 Prompt Example: F; D.4.2 State Difference Prediction

  • The full simulator prompt accounts for action and time-driven state changes, score rules, and UUID assignment; its difference variant returns changed or added objects, removals, and score updates.The difference output's score dictionary records score, game-over, and win status when relevant; otherwise it is empty.Source: §D.4.1 · p. 11, §D.4.2 · p. 11

D.5 Other Examples

  • Examples show the formats for action, object, and score rules, as well as state and state-difference JSON representations.The examples include a container's properties, a “put” action's constraints, and dish-cleaning rewards.Source: §D.5 · p. 12

E GPT-3.5 results

GPT-3.5 results show a large performance gap relative to GPT-4, with the difference larger when rules are absent. The authors characterize GPT-3.5 as especially weak at applying common-sense knowledge in this few-shot simulation task.

E GPT-3.5 results

  • The authors report a huge performance gap between GPT-4 and GPT-3.5 on object-property and game-progress prediction.Tables 5 and 6 report these respective outcomes.Source: E GPT-3.5 results · p. 12
  • The performance difference between GPT-4 and GPT-3.5 is larger when no rules are provided.Source: E GPT-3.5 results · p. 12
  • The authors describe GPT-3.5 as especially weak at applying common-sense knowledge in this few-shot world-simulation task.Source: E GPT-3.5 results · p. 12

F Histograms

The appendix points to detailed prediction results across model versions and output formats, and includes a table describing properties referenced in Figure 2.

F Histograms

Loading 2406.06485v1…