PaperlayerView exact revision on arXiv

Source-linked AI summary

From Traces to Agentic Worlds: Agentic Language World Models for Interactive Environment Simulation

Quanyu Long, Xiao Chen, Jianda Chen, Haozhen Zhang, Qisheng Hu, Jianzhu Bao, Wenya Wang

arXiv:2610.06100v1cs.CL

TL;DR

When original environments are inaccessible or impractical to reproduce, historical interactions offer a basis for recovering reusable environment knowledge. Trace2Env turns traces into a worldbook for an agentic world model, and across nine environments improves fidelity and multi-turn consistency over prompt-based LWMs.

  • Problem

    Realistic environment replicas are valuable for agent training and evaluation, but reproducing systems may require unavailable code, data, or infrastructure, while ordinary prompting is poorly suited to preserving long-horizon state consequences.

    Source: §1 · pp. 1–2

  • Method

    Trace2Env reconstructs historical traces into a reusable worldbook that a world model agent consults with persistent episode state and memory to predict observations and lasting state effects.

    Source: §1 · p. 2, §4.2 · p. 6

  • Results

    Across nine environments, Trace2Env improves next-observation fidelity and long-horizon interaction consistency over prompt-based LWMs, with simulated task-agent behavior remaining valid more often when replayed in the real environment.

    Source: §1 · p. 3, §7 · p. 12

  • Takeaways & Limitations

    Agentic language world modeling is an alternative direction for building realistic environment replicas without reconstructing the original executable system.

    Source: §1 · p. 3, §7 · p. 12

    Simulation is limited to observable behavior recovered from traces: the original transition function, implementation, and hidden state are unavailable during simulation.

    Source: §3.2 · p. 4

Abstract

from arXiv · show

Realistic environment replicas are increasingly valuable for training and evaluating LLM agents, yet the original systems may be inaccessible or impractical to reproduce. We explore agentic language world modeling: rather than rebuilding an executable environment, a world model agent serves as the environment for a task agent and supports faithful and stateful simulation. We instantiate this paradigm with Trace2Env, a learning-free framework for settings where the original system is unavailable but historical interaction traces remain accessible. Trace2Env reconstructs these traces into a reusable environment worldbook containing environment schemas, grounded evidence, and induced behavioral knowledge. At runtime, the world model agent actively consults the worldbook together with persistent episodic state to infer each action's observation and lasting state effects. Across nine environments, Trace2Env improves both next-observation fidelity and long-horizon interaction consistency over conventional prompt-based LWMs. In multi-turn interaction, task agent actions generated against Trace2Env remain valid more often when replayed in the real environment, indicating that its simulated dynamics better preserve the consequences of earlier actions across successive turns. These results establish agentic language world modeling as an alternative direction for building realistic environment replicas without reconstructing the original executable system.

1. Introduction

The introduction motivates reconstructing environments from historical traces when original systems cannot be reproduced, and proposes an agentic world model that uses a reconstructed worldbook and persistent episode context. It reports improved next-observation fidelity and multi-turn consistency over conventional prompting.

1. Introduction

  • Historical interactions can remain available even when reproducing a terminal, workspace, or internal system would require inaccessible code, data, or infrastructure.The traces record actions, observations, failures, and consequences of earlier actions.Source: §1 · p. 1
  • Long-horizon simulation must infer and preserve latent state changes, because an action such as deleting a file may have consequences only in a later turn.Flattening environment knowledge and interaction history into a prompt is described as poorly suited to reliable multi-turn consistency.Source: §1 · p. 2
  • Trace2Env treats environment simulation as an agentic task: a world model agent seeks relevant trace-derived knowledge and reasons over it with episode state and memory.The task agent chooses actions, while the separate world model agent determines the simulated environment response without direct access to the original environment.Source: §1 · p. 2
  • Its reusable worldbook combines schemas, observed demonstrations, and induced behavioral rules; at runtime, the agent proposes observations and persistent state changes for a shared harness to check and commit.The worldbook is a trace-grounded behavioral resource, not an executable reconstruction, and can support multiple episodes and runtime backbones.Source: §1 · pp. 2–3
  • Across nine environments, Trace2Env improves all five next-observation dimensions over conventional LWM prompting and better preserves behavior when simulated actions are replayed in the real environment.The evaluation covers format, factuality, consistency, realism, and overall quality.Source: §1 · p. 3

2. Related Work

Related work spans language world models that predict action consequences, methods that distill trajectories into reusable task knowledge, and executable environment reconstruction. Trace2Env instead uses a non-executable worldbook that an agentic LWM inspects during simulation.

2. Related Work

  • Trace2Env differs from RAWM and WorldEvolver by organizing traces into an explicit worldbook for selective inspection, and it studies simulation without access to the original environment.WorldEvolver assumes continued outcomes from the original environment to update its memories and rules.Source: §2 · p. 3
  • Trajectory-distillation methods produce reusable skills or workflows for task-agent decisions, whereas Trace2Env reconstructs environment behavior for an agentic LWM to simulate.The cited examples include Agent Workflow Memory, Trace2Skill, Memory–Skill Co-Evolution, SkillGen, and TraceCompiler (Wang et al., 2025; Ni et al., 2026; Tang et al., 2026; Ma et al., 2026; Yadouni and Li, 2026).Source: §2 · pp. 3–4
  • Terminal-Universe reconstructs executable terminal workspaces for real tool feedback and task synthesis, while Trace2Env uses a non-executable worldbook to simulate responses to subsequent actions.Terminal-Universe is cited as Wu et al. (2026).Source: §2 · p. 4

3. Preliminaries

The preliminaries describe language world models as predictors of environment observations after task-agent actions, then formalize trace-based reconstruction as learning a simulator from recorded interactions without reproducing the original system.

3.1. Language World Models

  • A language world model approximates the environment’s action–observation interface: the task agent selects an action, and the LWM predicts the next observation used in continuing interaction.The history includes prior observations and actions; observations may omit state changes, such as a file deletion.Source: §3.1 · p. 4

3.2. Problem Formulation

  • Trace-based reconstruction maps recorded interaction sequences into a reusable simulator that generates observations for new interactions.The original transition function, implementation, and hidden state are unavailable during simulation, and the LWM parameters remain fixed.Source: §3.2 · p. 4
  • The simulator’s objective is to approximate observable environment behavior under new action sequences without reproducing the original implementation.It predicts an observation from the simulated history, current action, and build traces, then appends that prediction to the simulated history.Source: §3.2 · p. 4

4. Trace2Env

Trace2Env turns historical interactions into reusable environment knowledge, then combines that fixed knowledge with episode-specific state and memory to simulate successive actions. Its world model agent proposes responses and state changes, while a shared harness validates and commits transitions.

4. Trace2Env

  • Trace2Env reconstructs build traces into an immutable worldbook and combines it with mutable episode state and interaction memory during online rollouts.A world model agent proposes transitions; a shared harness controls which state changes are committed.Source: §4 · pp. 4–5

4.1. Reconstructing an Environment Worldbook

  • The worldbook retains schemas, recorded transitions and demonstrations, induced behavioral abstractions, and provenance linking artifacts to supporting traces.Construction aligns actions with outcomes, withholds ambiguously attributed effects, and reviews proposed rules against supporting and contrasting cases.Source: §4.1 · pp. 5–6

4.2. A Persistent Episode Workspace

  • The persistent workspace separates reusable environment behavior from current episode facts and earlier interaction evidence; missing state entries remain unknown, not evidence of absence.The worldbook is shared and immutable, represented state is mutable and episode-specific, and memory preserves earlier turns and their temporal context.Source: §4.2 · p. 6

4.3. Agentic Simulation

  • For each action, the world model agent inspects relevant workspace resources and proposes an observation plus ordered candidate state effects, without modifying the episode directly.Retrieved evidence is classified as supporting, uncertain, or format-only so episode-specific facts from other traces are not assumed applicable.Source: §4.3 · pp. 6–7
  • The shared harness checks proposals against schemas, rule support, and applicable constraints, commits only accepted changes, and carries updated state and memory into the next turn.The returned observation is resolved under applicable rendering constraints.Source: §4.3 · p. 7

5. Experiments

Across nine environments, Trace2Env improves next-observation prediction over Direct Prompting and produces more real-environment-consistent multi-turn behavior. Ablations indicate that evidence and abstraction contribute complementary benefits, while additional construction traces trade higher prediction scores for greater one-time cost.

5.1. Experimental Setup

  • Evaluation covers seven next-observation environments and two multi-turn text games, using separate protocols for prediction and replayed-action success.Next-observation predictions receive five scores; multi-turn evaluation reports Real, WM, W2R, and CR = W2R/Real.Source: §5.1 · p. 7
  • Worldbooks use environment-specific trace sources and held-out evaluation data, while five compared systems test prompting, retrieved traces, static worldbooks, the harness, and full Trace2Env.Worldbooks are reconstructed with gpt-5.6-sol and kept fixed across model backbones.Source: §5.1 · p. 8

5.2. Next-Observation Prediction Results

  • 75.94 with gpt and 75.75 with deepseek: Trace2Env achieves the highest average score, improving over Direct Prompting by 6.43 and 7.04 points, respectively.It is best in 11 of 14 environment–backbone pairs and never underperforms Direct Prompting.Source: Table 1 · p. 8, §5.2 · p. 9

5.3. Long-horizon Interaction Consistency

  • Trace2Env yields higher real-environment replay consistency than Direct Prompting by preserving state consequences, even when rejecting actions lowers apparent simulated success.For an unsupported put action, Trace2Env predicts a no-op and preserves inventory state, while Direct Prompting hallucinates a successful placement.Source: §5.3 · p. 9, Table 2 · p. 9, Figure 4 · p. 11

5.4. Ablations

  • On Terminal, evidence improves the schema-only variant by +2.45±0.89; abstraction alone does not help but adds +0.69 ± 0.63 on top of evidence.Adding evidence to an abstracted worldbook gives +4.26 ± 1.02, and the full worldbook performs best; the table reports paired score differences with standard errors.Source: §5.4 · p. 10, Table 3 · p. 10
  • On Terminal, prediction rises from 64.89 at 20 traces to 66.42 at 36, while one-time construction cost also increases.The curve is still rising at 36 traces, the largest available non-overlapping set; the main experiments use 20 as a practical quality-cost trade-off.Source: §5.4 · p. 10, Figure 3 · p. 10

6. Qualitative Results

A single unsupported transition can send a simulated rollout down a path the real environment never reached; Trace2Env instead preserves the no-op and the unchanged inventory state, allowing recovery.

6. Qualitative Results

  • Trace2Env predicts the unsupported placement as a no-op, keeps the tissuebox in inventory, and preserves that state so later responses support recovery.Direct Prompting commits a false placement and produces a rollout that appears successful in simulation but fails when replayed in the real environment.Source: §6 · p. 11

7. Conclusion

Trace2Env uses a trace-reconstructed worldbook, persistent episode state, and episodic memory to simulate an environment for a task agent. Across nine environments, it improves next-observation fidelity and long-horizon interaction consistency over prompt-based LWMs, while remaining bounded by the evidence traces reveal.

7. Conclusion

  • Trace2Env serves as the task agent’s environment, using a trace-reconstructed worldbook with persistent episode state and episodic memory.Its world model agent actively uses these resources during simulation.Source: §7 · p. 12
  • Across nine environments, Trace2Env improves next-observation fidelity and long-horizon interaction consistency over prompt-based LWMs.Component studies show complementary contributions from reconstructed knowledge and agentic operation.Source: §7 · p. 12
  • Trace2Env’s fidelity is bounded by what the traces reveal, and some backbones rely on reconstructed evidence less consistently than others.The paper identifies strengthening worldbook inspection and inference and expanding reconstruction beyond observed behavior as future directions.Source: §7 · p. 12

Appendix Contents

  • Table 4 summarizes environments, evaluation data, and construction data; cross-fitted worldbook statistics are reported as ranges over per-fold worldbooks.Source: Table 4 · p. 15

A. Experimental Setup

The evaluation covers next-observation prediction and multi-turn interaction across nine environments, with construction data drawn independently where possible and cross-fitting used where original environments are unavailable. Comparisons and uncertainty estimates are designed to isolate trace-derived knowledge and runtime contributions while controlling data overlap.

A. Experimental Setup

  • Evaluation uses AgentWorldBench and EnvScaler records for next-observation prediction and ALFWorld and SciWorld for multi-turn interaction.AgentWorldBench uses official test records; trajectories, rather than individual records, are the statistical units, and Android trajectory boundaries are recovered from record prefixes.Source: A. Experimental Setup · pp. 15–16
  • Terminal and Web use independent construction trajectories, while Android and SWE use task-grouped five-fold cross-fitting because their original environments are unavailable.EnvScaler construction and evaluation trajectories are disjoint by task; ALFWorld worldbooks use 40 simulator-confirmed successful trajectories, excluding failed attempts and hidden simulator state.Source: A. Experimental Setup · pp. 15–16
  • Comparisons include direct prompting, harness-only, raw-trace retrieval, and static-worldbook prompting; Trace2Env and harness-only share the same agent budget and official input.Paired differences use a trajectory-clustered bootstrap with 10,000 resamples and 95% percentile confidence intervals; intervals containing zero are not treated as statistically distinguishable. The evaluation pipeline prevents target observations from entering construction data.Source: A. Experimental Setup · pp. 16–17

B.1. Full Results Across Environments and Backbones

Trace2Env improves aggregate scores over Direct Prompting across all environment–backbone pairs, with gains spanning multiple judge dimensions. Paired comparisons find positive contributions from both reconstructed knowledge and runtime operation, though their relative importance varies by environment.

B.1. Full Results Across Environments and Backbones

  • Tables 5 and 6 decompose aggregate scores into five judge dimensions across seven next-observation environments; Table 5 marks the best total per environment in bold.Source: §B.1 · p. 17, Table 5 · p. 18
  • Trace2Env improves aggregate scores over Direct Prompting in all 14 environment–backbone pairs; factuality, consistency, realism, and quality also increase in every pair.Format, factuality, consistency, realism, and quality all improve simultaneously in 12 of 14 pairs; in the other two, only format decreases slightly.Source: §B.1 · p. 17

B.2. Paired Row Differences

  • The worldbook’s mean effect over Harness only is positive in all eight comparisons and statistically clear in six; runtime gains are strongest on Terminal and SWE.Trace2Env improves over Direct Prompting in every environment–backbone pair, with all trajectory-clustered confidence intervals excluding zero.Source: §B.2 · p. 17
  • Static Worldbook Prompting is competitive with or better than raw Trace RAG Prompting in seven of eight comparisons, while full Trace2Env generally improves further over both single-call alternatives.Source: §B.2 · pp. 17–18

B.3. Unseen-Task Transferability

Trace2Env’s gains extend beyond tasks represented in construction traces, though relevant trace coverage can add benefit. Overlap is evaluated using environment-specific units.

B.3. Unseen-Task Transferability

  • Overlap splits use environment-specific units—task, template, app, or repository—and Table 8 distinguishes gains against Direct Prompting from gains against Harness only.Source: §B.3 · p. 18, Table 8 · p. 20
  • No-overlap gains remain clear on Web and SWE under both backbones, and on Terminal with GPT, but are smaller with DeepSeek.Table 8 reports worldbook gain by construction overlap.Source: §B.3 · p. 19, Table 8 · p. 20
  • Relevant construction coverage can add benefit, most clearly for same-task Terminal examples and Android apps represented elsewhere in construction.The passage characterizes the worldbook as capturing reusable behavior while still benefiting from broader trace coverage.Source: §B.3 · p. 19, Table 8 · p. 20

B.4. Complete Terminal Ablations

Terminal ablations indicate that both worldbook contents and runtime components contribute to performance. Lighter runtime modes trade some fidelity for lower inference cost.

B.4. Complete Terminal Ablations

  • 64.18 for the evidence-enabled worldbook exceeds Trace RAG Prompting’s 58.42, and the full worldbook achieves the best score.Abstraction alone does not improve the schema-only system but becomes beneficial when paired with evidence.Source: §B.4 · p. 20, Table 9 · p. 21
  • Table 9’s Terminal ablations use 354 evaluation records and the gpt-5.6-sol backbone, varying worldbook knowledge components and runtime components.Worldbook variants add abstraction or evidence over schema-only; agent-side variants remove the adaptive loop or state tracking.Source: §B.4 · p. 20, Table 9 · p. 21

B.5. Efficiency–Accuracy Trade-off

  • Removing the adaptive loop retains about 80% of Trace2Env’s improvement over Direct Prompting at 29% of its inference cost; removing state tracking saves 23% of cost with a one-point score decrease.The full runtime has the highest fidelity, and worldbook construction is paid once per environment and reused.Source: Table 10 · p. 21, §B.5 · p. 21

B.4. Complete Terminal Ablations

  • Removing the adaptive loop reduces performance by 1.79 ± 0.91, while removing persistent state tracking reduces it by 0.97 ± 0.78; these results indicate both runtime components contribute.Source: §B.4 · p. 21

C. Implementation Details

The appendix details worldbook artifacts, their reconstruction, the simulation harness, reported settings, and representative case studies.

C. Implementation Details

  • The appendix supplements Section 4 with details on worldbook components, reconstruction, the simulation harness, and reported settings.Source: C. Implementation Details · p. 21

C.1. Worldbook Artifacts

  • Worldbook schemas organize durable world facts, session context, visible surface state, and uncertain or conflicting epistemic beliefs; evidence preserves aligned transitions and abstractions encode reviewed rules and constraints.Artifacts and demonstrations retain provenance to supporting episodes and turns.Source: §C.1 · pp. 22–23

C.2. Reconstructing the Worldbook

  • Reconstruction aligns actions with results, extracts local evidence, induces schemas, and proposes abstractions reviewed against evidence; only supported rules meeting compilation criteria become executable.Ambiguously attributed or concurrent effects are withheld, and the resulting worldbook stays fixed during simulation.Source: §C.2 · p. 23

C.3. The Simulation Harness

  • For each action, the harness retrieves and assesses worldbook entries, builds a brief from action, state, memory, and evidence, validates proposed effects, renders an observation, then commits state and memory.Rejected proposals use an observation-only fallback under the reference policy; current state and episode memory take precedence over retrieved examples.Source: §C.3 · pp. 23–24

C.4. Hyperparameters and Settings

  • Reported-run settings parameterize or bound reconstruction and harness mechanisms; harness calls use no sampling temperature, while the prompting baseline and judge use 0.6.Source: §C.4 · p. 24, Table 13 · p. 24

D. Case Study

  • Three case studies use actual predictions, retrieved evidence, and tracked state to examine environment conventions, accumulated state, and recovery in a multi-turn ALFWorld episode.Table 14 organizes each case by the information needed for faithful prediction, what Direct Prompting misses, and what Trace2Env uses.Source: Table 14 · p. 25, D. Case Study · p. 25

D.1. Single-Turn Prediction on Terminal

This section describes Trace2Env’s single-turn terminal prediction process, including single-call baselines, trace-derived worldbook construction, and runtime use of applicable trace evidence with persistent state. It also presents terminal examples and documents the experimental prompts and Direct Prompting interface.

D.1. Single-Turn Prediction on Terminal

  • 90 versus 30 judge scores show Trace2Env reconstructing a complete heredoc interaction from cross-task examples and an induced Bash continuation-prompt convention.The current episode contained no prior heredoc; Direct Prompting omitted the initiating command and much of the deterministic interaction.Source: §D.1 · pp. 25–26
  • 100 versus 55 judge scores show Trace2Env reproducing pandas, python-dateutil, and six versions, while Direct Prompting omits dependencies and predicts an incorrect version.Trace2Env combines episodic memory, tracked package-version state, and a retrieved installation trace to infer the later install result.Source: §D.1 · p. 26

D.2. Multi-Turn Interaction on ALFWorld

  • Trace2Env preserves a failed ALFWorld placement, exposes valid commands through subsequent observations, and enables a recovery sequence that succeeds on real replay with reward 1.Direct Prompting instead treats the unsupported placement as successful, producing an internally consistent rollout that fails in the real environment.Source: §D.2 · pp. 27–28

E. Prompts

  • The appendix includes the official AgentWorldBench prompt, single-call baseline context, worldbook-construction and runtime prompts, while leaving unchanged official judge prompts unreproduced.Source: E. Prompts · p. 28

E.1. Direct Prompting: the Official AgentWorldBench System Prompt

  • Direct Prompting receives the environment-specific system prompt, full trajectory prefix, and current action to predict the next observation; Trace2Env receives the same official environment description.The Terminal prompt shown is abridged, with repetitive command-specific sections and additional few-shot examples omitted.Source: §E.1 · p. 28
  • The direct-prompting terminal example shows `aws --version` returning an AWS CLI version string in the environment observation.Source: §E.1 · pp. 31–32

E.2. Single-Call Baselines

  • Trace RAG Prompting appends retrieved raw action–observation turns, whereas Worldbook Prompting appends a static reconstructed package with schemas, induced knowledge, demonstrations, and retrieved evidence.Source: §E.2 · p. 32

E.3. Worldbook Construction

  • Worldbook construction proceeds through transition extraction, schema induction, rule induction and review, renderer-contract induction, and note induction.Stages consume trace evidence or structured outputs from preceding stages.Source: §E.3 · p. 33
  • Rule induction derives reusable, provenance-supported transition rules with conditions and effects, while a falsification review checks them against transitions and downgrades rules when evidence conflicts.Rules should capture behavior shared across episodes rather than replaying a single invocation.Source: §E.3 · pp. 34–35

E.4. Runtime: World Model Agent and State Tracker

  • The world model agent receives the current action, episode state, relevant worldbook knowledge, and interaction history, then submits a typed prediction of the observation and state effects.Workspace tools expose reconstructed knowledge, persistent state, and episodic memory.Source: §E.4 · pp. 36–37
  • Trace packages preserve raw action–observation turns and provide tools to retrieve similar turns, search demonstrations or evidence, and read exact observations with neighboring actions.Retrieved turns can inform output formats, prompt conventions, error shapes, and program behavior, but episode-specific files, values, and identifiers must come from the current episode.Source: §E.4 · p. 38
  • An applicability gate distinguishes intact supporting evidence from sanitized format-only material, and abstains from package evidence when no item supports the transition.Supporting items share files or operands but yield to the current transcript and state; uncertain or format-only items use placeholders that must not be copied or replaced with invented values.Source: §E.4 · pp. 38–39
  • The state tracker updates persistent session state with minimal typed mutations supported by observed actions and environment responses, using declared schema paths and marking indeterminate paths uncertain.Object-valued fields are edited with merge or remove operations rather than replaced wholesale.Source: §E.4 · p. 39
  • Wait observations record program completion or continued execution, produced files and results, and the returned prompt; displayed exact values are stored verbatim, with truncated contents marked incomplete.Unmentioned entries remain unknown rather than being deleted or blanked.Source: §E.4 · p. 39
Loading 2610.06100v1…