PaperlayerView exact revision on arXiv

Source-linked AI summary

PAN: A World Model for General, Actionable, and Long-Horizon World Simulation

PAN Team, Zihan Liu, Yi Gu, Mingkai Deng, Guangyi Liu, Zeyu Feng, Qiyue Gao, Yiyan Hu, Benhao Huang, Yichi Yang, Kun Zhou, Jiannan Xiang, Zhiting Hu, Zhengzhong Liu, Eric P. Xing

arXiv:2511.09057v4cs.CVcs.AIcs.CLcs.LG

TL;DR

Visual realism alone does not establish world-model competence, and current approaches remain limited in action-driven prediction and long-horizon consistency. PAN applies the GLP architecture to world simulation and reports the highest overall performance among open-source world models, with competitiveness against leading commercial systems on action-conditioned simulation and long-horizon forecasting.

  • Problem

    Visual realism alone does not establish world-model competence, and standard video-generation metrics do not measure causal action effects or consistency across longer rollouts.

    Source: §1 · p. 2, §3 · p. 5

  • Method

    PAN uses GLP to predict compact latent world states and generate causally conditioned observation chunks for rollout simulation.

    Source: §1 · p. 2, §5.1 · p. 10, §5.2 · p. 11

  • Results

    PAN achieves the highest overall performance among open-source world models and is competitive with leading commercial systems on action-conditioned simulation and long-horizon forecasting.

    Source: §1 · p. 3

  • Takeaways & Limitations

    World-model evaluation should directly assess action simulation, long-horizon consistency, and planning rather than rely on perceptual realism metrics alone.

    Source: §3 · p. 5

    PAN can still exhibit semantic drift, object inconsistency, visual artifacts, and reduced action controllability in extended or out-of-distribution rollouts.

    Source: §7 · p. 35

Abstract

from arXiv · show

A world model is a cognitive simulator of the real-world environment allowing biological agents to reason about how the world evolves, whether spontaneously or in response to their actions, and accordingly to plan and strategize. In building Artificial Intelligence (AI) systems, world models represent the next frontier beyond large language models (LLMs) to enable physical and embodied intelligence in AI agents, allowing them to perform decision-making through simulative reasoning and reinforcement-learning through simulative trials. Recent advancements in world modeling have yielded impressive progress in video generation, 3-D scene evolution, robotic dynamics, and game simulation, but limitations persist in general, open-domain, action-driven prediction, long-horizon consistency, and abstract reasoning and planning. Moreover, fundamental architectural questions, whether it be state representation, information flow, or training objectives, remain unresolved. In this paper, we introduce PAN, a world model built on the Generative Latent Prediction (GLP) architecture. GLP combines stateful latent representations of world states; an encoder--decoder closed-loop information flow; an LLM/diffusion-based mixed reasoning backbone; and a non-degenerate generative reconstruction objective whose fidelity is ``dampable'' to balance fine-grained detail against semantic saliency. Compared to several existing systems, PAN demonstrates advantages beyond standard video generation in action-conditioned world simulation, long-horizon forecasting, and simulative reasoning and planning, capabilities we argue should serve as the primary criteria for evaluating world models.

1 Introduction

PAN is introduced as a general world model built on Generative Latent Prediction (GLP) to address limits in open-domain action-driven simulation, long-horizon consistency, and reasoning. The introduction reports benchmark and planning-agent results and describes a reconstruction-grounded architecture.

1 Introduction

  • A world model predicts action-conditioned next states, enabling agents to compare hypothetical plans; classical models showed planning and sample-efficiency benefits but remained task-specific.Examples include learned dynamics models (Ha and Schmidhuber, 2018), Dreamer (Hafner et al., 2019a, 2023), and discrete-token predictors (Micheli et al., 2022).Source: §1 · p. 1
  • Existing approaches leave open-domain, action-conditioned temporal consistency unresolved, with limitations spanning narrow action spaces, weakly grounded latent prediction, and video generation without persistent state or control.Video generators can be visually realistic but typically lack persistent internal states, controllable action interfaces, and object-level causal dynamics (Yang et al., 2024b; Kong et al., 2024; Wan et al., 2025). Domain-specific systems have limited generality, while JEPA-style latent prediction can lose relevant information and symbolic reasoning; 3D models often lack temporal dynamics and meaningful actions beyond navigation (Micheli et al., 2022; Alonso et al., 2024; Bruce et al., 2024; Wang et al., 2023b; Hu et al., 2023; Yang et al., 2024a; NVIDIA, 2025; LeCun, 2022; Assran et al., 2025; Xing et al., 2025; World Labs, 2024).Source: §1 · p. 2
  • GLP couples an observation encoder, action-conditioned latent predictive backbone, and decoder trained to reconstruct observations, grounding predictions and discouraging latent collapse or information loss.Because discarded detail creates reconstruction error, the encoder is pressured to preserve dynamically relevant information while the backbone models abstract causal dynamics and the decoder handles perceptual realization (Xing et al., 2025).Source: §1 · p. 2
  • PAN uses an LLM reasoning backbone and Causal Swin-DPM to support symbolic, long-horizon reasoning while dampening incidental pixel detail in generative reconstruction.Causal Swin-DPM conditions each chunk on partially noised preceding chunks, balancing fine-grained fidelity against semantically persistent structure; PAN trains on 8 million video–text pairs with dense temporal captions and shows rollouts over 10+ action steps.Source: §1 · pp. 2–3
  • PAN achieves the highest overall performance among open-source world models, competes with leading commercial systems on simulation and forecasting, and improves planning-agent task success over both baselines.The benchmark covers action-conditioned simulation, long-horizon forecasting, and simulative reasoning and planning.Source: §1 · p. 3

2 PAN architecture

PAN uses Generative Latent Prediction (GLP) to evolve latent world states under actions and decode them into video observations. Its reconstruction-based supervision grounds latent transitions in realizable sensory changes, while its implementation supports language-conditioned video-chunk rollouts.

2.1 Generative latent prediction

  • PAN encodes video chunks into latent states, predicts each next state from state–action history, and decodes it into temporally coherent video segments.The decoder also uses recent observation context for local visual continuity, while the latent state remains the primary carrier of action-conditioned dynamics.Source: §2.1 · pp. 3–4, Figure 1 · p. 4

2.2 Generative supervision

  • Reconstruction-based supervision requires each predicted latent transition to yield an observation, grounding learned dynamics in actionable, evidence-aligned sensory change.Unlike encoder-only latent matching, which can admit degenerate solutions and loosely proxy observation-level consistency, GLP decodes predictions into the sensory domain (Assran et al., 2023; Xing et al., 2025).Source: §2.2 · pp. 4–5

2.3 Design choices and parameterization

  • PAN combines a Qwen2.5-VL vision encoder and autoregressive backbone with a Wan2.1 video diffusion decoder and Causal Swin-DPM for chunk-wise denoising.The backbone predicts states from latent-state and language-action history; the decoder’s sliding temporal window supports smooth segment transitions while causal masking prevents future-action leakage (Feng et al., 2024).Source: §2.3 · p. 5
  • Actions are natural-language instructions at the video-chunk level; low-level continuous motor-control adaptation is left to future work.The discrepancy function uses flow matching (Lipman et al., 2022), and GLP can be extended with hierarchical embeddings, mixed discrete–continuous representations, or higher-order temporal dynamics.Source: §2.3 · p. 5

3 Evaluation on world reasoning

PAN is evaluated as a simulator for action consequences, coherent extended rollouts, and planning—not only for perceptual realism. WR-Arena measures these capabilities across action simulation fidelity, long-horizon forecasting, and simulative reasoning and planning.

3 Evaluation on world reasoning

  • PAN achieves the highest overall WR-Arena scores among open-source models and is competitive with commercial systems across the three evaluated dimensions.Figure 2 summarizes performance on Action Simulation Fidelity, Long-Horizon Forecast, and Simulative Planning.Source: Figure 2 · p. 6

Action Simulation Fidelity

  • PAN scores 70.3 on agent simulation and 47.1 on environment simulation, for an overall action-fidelity score of 58.7, highest among open-source models and comparable to the strongest commercial systems.Entity-level actions are better specified than scene-level interventions; PAN uses step-wise, action-conditioned rollouts, unlike video-generation baselines that cannot condition each segment on the preceding action’s outcome.Source: §3 · p. 7

Long-Horizon Forecast

  • PAN scores 53.6% on Transition Smoothness and 64.1% on Simulation Consistency, the highest scores among all evaluated models, including commercial systems.Causal Swin-DPM provides overlapping denoising context across chunks and accounts for a 5.7-point Transition Smoothness advantage over the next-best model; PAN’s consistency degrades gradually over 9 rollout steps, while standalone generators deteriorate rapidly. Ablations also attribute additional gains in agent dynamics, transition smoothness, and consistency to the VLM-based backbone.Source: §3 · p. 7

Simulative Reasoning and Planning

  • PAN achieves 56.1% on Step-Wise Simulation, highest among open-source baselines, while its VLM-agent integration raises Open-Ended Planning and Structured Planning success by 26.7% and 23.4%, respectively, versus no world model.The planning agent simulates candidate-action consequences and chooses the predicted outcome that best advances its goal; PAN’s observation-space reconstruction training makes predictions causally informative, while other models can sometimes mislead the planner.Source: §3 · pp. 7–8

4 Discussion

PAN frames world models as simulators for an agent’s hypothetical reasoning, not merely media generators (Xing et al., 2025). It keeps observation-grounded state prediction separate from goal-directed action selection.

4 Discussion

  • PAN separates a world model’s observation-grounded next-state predictions from an agent model’s goal-directed action selection.The world model learns from observed transitions, while the agent model learns from reward feedback; combining them can bias predictions toward rewarding outcomes and undermine action selection. Prior simulative-reasoning work found separate world models outperformed reactive baselines and monolithic approaches (Hao et al., 2023; Deng et al., 2025).Source: §4 · p. 9
  • GLP combines stateful symbolic action-conditioned latents, an LLM–diffusion backbone, and decoder-based generative supervision for structured long-horizon simulation.Removing the VLM backbone preserves short-horizon fidelity but degrades long-horizon consistency and agent dynamics; full GLP sustains coherent extended rollouts where standalone generators degrade rapidly.Source: §4 · p. 9
  • Generative supervision anchors latent predictions to observable reality, and its loss lower-bounds latent loss, whereas good latent predictions need not reconstruct well (Theorem 2, Xing et al., 2025).PAN’s partially noised preceding-chunk context dampens incidental pixel detail; its reconstruction loss decreases smoothly across both training stages, while latent-only objectives require collapse-preventing regularizers with optimization and hyperparameter costs.Source: §4 · pp. 9–10
  • FID and FVD miss simulator-relevant causal fidelity, action controllability, and long-horizon consistency; WR-Arena (Gao et al., 2026) measures action simulation, forecasting, and planning.VLM-judge bias, human-annotator variability, and a lack of shared evaluation platforms remain concerns; future benchmarks should include horizon-dependent degradation, causal counterfactuals, and standardized planning protocols.Source: §4 · p. 10
  • PAN’s language-level action interface limits fine manipulation, while long-horizon drift and inherited perceptual rather than physical priors remain unresolved.Fine manipulation requires continuous torque commands and closed-loop proprioceptive feedback; PAN inherits visual priors from Wan 2.1, which may yield plausible but causally incorrect interactions. Proposed directions include multi-resolution actions, richer stateful representations, diverse data, and simulation-focused training from scratch.Source: §4 · p. 10

5 Methods

PAN implements GLP through an autoregressive latent-state predictor coupled to generative video reconstruction, with causal chunk-wise denoising designed to support long rollouts. Its methods also specify a two-stage training pipeline, data construction, evaluation protocol, and autoregressive inference.

5.1 Model implementation

  • PAN predicts each next latent state from visual-state tokens, a natural-language action, and 256 query embeddings, using a 256-token bottleneck and a generative decoder to preserve information needed for faithful simulation.The pretrained multimodal backbone also supplies language-conditioned sequence modeling and compositional reasoning; the video decoder combines state context with action-specific conditioning and anchors latent transitions to reconstructed observations.Source: §5.1 · pp. 10–11

5.2 Causal Swin-DPM

  • Causal Swin-DPM denoises two chunks at different noise levels, with causal attention preventing future-action leakage and partially noised context emphasizing persistent semantics over incidental pixel detail.With K = 1000 denoising steps, the earlier chunk starts at K/2 and is dequeued after K/2 steps as a new noise chunk is added; training enforces the K/2 gap between chunk noise levels.Source: §5.2 · p. 11

5.3 Training

  • Training proceeds in two stages: decoder adaptation with the pretrained encoder and backbone frozen, followed by end-to-end generative-supervision training with the vision-language model frozen.Stage 1 runs for 5 epochs on 960 NVIDIA H200 GPUs; Stage 2 trains only query embeddings and the video decoder, limits history to 10 rounds, and uses early stopping after 1 epoch.Source: §5.3 · p. 11

5.4 Training data

  • The training-data pipeline filters publicly sourced videos across human activities, interactions, scenes, navigation, and multi-agent scenarios, retaining 16.1% of clips and producing 8 million video–text pairs with temporal-dynamics captions.The pairs represent approximately 4,500 hours of video; long videos are segmented at shot boundaries before filtering.Source: §5.4 · pp. 11–12

5.5 Evaluation protocol

  • The evaluation protocol measures action-simulation fidelity, long-horizon forecast continuity and consistency, and simulative reasoning and planning in robotic manipulation and tabletop scenarios.PAN is compared with open- and closed-source models, including WAN, Cosmos, V-JEPA 2, KLING, MiniMax, and Gen-3.Source: §5.5 · p. 12

5.6 Inference

  • At inference, PAN autoregressively predicts future states and reconstructs observations, then combines reconstructed-observation encoder outputs with predicted states to improve long-term consistency.Decoding uses classifier-free guidance at scale 4; SageAttention2++ achieves a reported 30.3% speedup with minimal quality impact.Source: §5.6 · p. 12

5.7 Data availability

  • PAN’s training data comprises 8 million video–text pairs, approximately 4,500 hours of video, from publicly accessible sources.The paper directs readers to Supplementary Information §3 for data-construction details and statistics.Source: §5.7 · p. 12

Corresponding Author · Extended Data

PAN simulates action-conditioned video over long sequences, follows instructions across visually distinct worlds, and generates prompted rare or counterfactual scenarios.

Extended Data Fig. 1: Interactive long-horizon simulation

  • Starting from one frame, PAN generates consecutive video segments conditioned on a sequence of natural-language action instructions.The examples include multi-step interactive simulations over 10+ action steps.Source: Extended Data Fig. 1: Interactive… · p. 14
  • The long-horizon examples span kitchen manipulation and autonomous driving scenarios over 10+ action steps.Each simulation begins from a single initial frame and follows a sequence of action instructions.Source: Extended Data Fig. 1: Interactive… · p. 14

Extended Data Fig. 2: Diverse action and world simulation

Extended Data Fig. 3: Rare case simulation

1 Model Architecture Details

PAN encodes video into structured latent states, predicts future states from interaction history and actions, and decodes them into observations with a diffusion model. Its Causal Swin-DPM uses causal chunk-wise denoising context to support coherent action-conditioned rollouts.

1.1 Vision Encoder

  • PAN’s vision encoder converts frames into spatially and temporally organized visual tokens that serve as latent representations for the predictive backbone.It is adapted from Qwen2.5-VL-7B-Instruct; video frames use 3D patch partitioning to represent short-term motion and temporal coherence.Source: §1.1 · p. 19

1.2 Autoregressive World Model Backbone

  • The autoregressive backbone predicts each next latent state from the full interaction history and current natural-language action, using multi-turn exchanges that interleave visual states and actions.Each assistant turn emits the next state; 256 learnable query embeddings produce a fixed-length sequence of 256 continuous latent tokens for the decoder.Source: §1.2 · p. 19, §1.2.1 · pp. 19–20, §1.2.2 · p. 20

1.3 Video Diffusion Decoder

  • The Wan2.1-T2V-14B diffusion decoder generates the next video from the predicted latent state and action, combining dedicated world-state and text cross-attention streams.The world-state stream has a zero-initialized output projection; its output is summed with action-text conditioning from umT5.Source: §1.3 · p. 20, Figure S1 · p. 21

1.3.1 Flow Matching Objective

  • The flow-matching objective implements the GLP observation discrepancy using Rectified Flow, which defines a linear interpolation path between noise and the target observation.Training predicts the ground-truth velocity along that path.Source: §1.3.1 · p. 20

1.3.3 Causal Swin-DPM

  • Causal Swin-DPM uses partially noised previous-chunk context and chunk-wise causal attention to reduce error accumulation and discontinuities without leaking future actions.Its sliding window lets the current chunk attend to preceding denoising context, while masking content associated with unseen future actions.Source: §1.3.3 · pp. 21–23
  • Causal Swin-DPM denoises two video chunks at different noise levels in a sliding window, dequeuing the earlier chunk after K/2 steps and adding a new noisy chunk.For the model’s K = 1000 denoising steps, the earlier chunk begins at K/2 noise and the later chunk at K.Source: §1.3.3 · p. 22

1.3.4 Variational Autoencoder Padding

  • The Wan2.1 3D causal VAE compresses 1 + T frames into 1 + T/4 latent features, and training randomly pads each 81-frame clip with 0 to 122 preceding frames for temporal context.The decoder window size is 21 latent frames, corresponding to 81 real video frames; the added context frames are discarded after encoding.Source: §1.3.4 · p. 23

1.3.5 Conditioning Frame and Noise Augmentation

  • The sliding window uses the previous chunk’s last frame as its conditioning frame and adds Gaussian noise at k = 0.055 to mitigate long-horizon error accumulation.Training computes no loss on the conditioning frame.Source: §1.3.5 · p. 23

2 Training Details

PAN is trained in stages, first specializing modules and then integrating them under a generative objective; at inference, it feeds generated observations back into its state history for extended simulation.

2.1 Stage 1: Module-Wise Training

  • Module-wise training specializes components before integration: PAN retains its pretrained Qwen2.5-VL backbone and adapts Wan2.1-T2V into a causal decoder for temporally smooth, action-controlled simulation.The decoder’s linear weights are kept at zero during this stage, so it does not yet depend on latent world states.Source: §2.1 · p. 23

2.2 Stage 2: Joint Training

  • Joint training couples latent prediction to observation-space reconstruction, aiming for semantically and physically consistent outputs while reducing representation collapse from purely latent objectives.The discrepancy uses flow-matching loss rather than pixel-for-pixel reproduction. The vision-language model is frozen while query embeddings and the video diffusion decoder are trained; training uses observed chunks, with early stopping after one epoch, and inference feeds back generated observations.Source: §2.2 · pp. 24–25

2.3 Inference

  • PAN predicts future states sequentially from an initial observation and proposed actions, then augments predicted states with encodings of reconstructed observations to track the evolving world.The resulting state-and-action history conditions subsequent predictions, enabling rollout from generated rather than ground-truth observations.Source: §2.3 · p. 25

2.3.1 Causal Inference for Video Diffusion Decoder

  • Caching the current chunk’s denoising output avoids recomputing it as new chunks enter; classifier-free guidance uses a scale of 4 to improve generation quality.Once an earlier chunk is fully denoised and dequeued, its cached output is discarded.Source: §2.3.1 · p. 25

2.3.2 Inference Accelerations

  • 30.3% end-to-end speedup is achieved with negligible generation-quality impact using SageAttention2++ with 8-bit computation and 4-bit key–value quantization.Inference also uses sequence parallelism with a default group size of 8 and a 5:3 split between the first and second chunks.Source: §2.3.2 · pp. 25–26

3 Training Data Pipeline

PAN’s training data pipeline turns public YouTube videos into aligned video–text examples for modeling temporal world dynamics. It segments videos, filters clips for suitability, and adds dense captions focused on changes and actions.

3 Training Data Pipeline

  • The dataset draws on YouTube-8M (Abu-El-Haija et al., 2016) and YT-Temporal-180M (Zellers et al., 2021), spanning human activities, interactions, varied scenes, navigation, and multi-agent scenarios.Video and text are aligned so semantic descriptions accompany fine-grained visual dynamics.Source: §3 · p. 26

3.1 Video Segmentation

  • A three-stage shot-boundary strategy adapted from Panda-70M detects hard cuts and gradual transitions to produce coherent clips with temporal continuity.The stages are designed to identify both abrupt scene transitions and soft fades or dissolves.Source: §3.1 · p. 26

3.2 Data Filtering Pipeline

  • The filtering pipeline combines rule-based motion and quality checks, pretrained detector filters, and a custom vision–language model filter.These complementary stages target clips unsuitable for world-model training.Source: §3.2 · p. 26

3.2.1 Rule-Based Filters

  • Rule-based filters use optical-flow magnitude, edge difference, and luminance difference to reject clips with static, extreme, poorly distributed, or cut-dominated motion.They also remove clips with low average optical-flow magnitude.Source: §3.2.1 · p. 26

3.2.2 Pretrained Detectors

  • Pretrained detectors exclude clips below an aesthetic-score threshold and clips whose sampled frames exceed a threshold for text-covered area.The aesthetic scorer evaluates sampled frames, while MixNet detects text regions in start, middle, and end frames.Source: §3.2.2 · pp. 27–28

3.2.3 Custom VLM-Based Filter

  • The custom VLM filter targets lecture-style, text-dominated, screen-recording, visually degraded, and heavily edited clips.It is intended to identify low-quality categories that earlier filtering stages cannot reliably capture.Source: §3.2.3 · p. 28
  • The VLM was fine-tuned on human labels for 11,173 clips, with thresholds chosen to maximize validation F1 and clips discarded when any property exceeded its threshold.The sampled clips had passed the rule-based filters and pretrained detectors; the model was trained with a 9:1 train/validation split.Source: §3.2.3 · p. 28

3.3 Dense Video Captioning

  • Dense re-captioning produces 8 million video–text pairs from approximately 4,500 hours of video, with descriptions focused on temporal dynamics rather than generic summaries or static backgrounds.The captioning requirements emphasize factual detail and motion, events, environmental changes, and emerging objects, following findings from DALL-E 3.Source: §3.3 · p. 28

4 Evaluation Details

Evaluation follows WR-Arena’s three-dimensional framework, assessing action simulation fidelity, long-horizon forecasting, and simulative reasoning and planning against a set of video-generation and world-model baselines.

4.1 Benchmark Specifications

  • The evaluation adopts WR-Arena’s three-dimensional framework; full protocols, metric definitions, dataset specifications, and code are available in Gao et al. (2026).Source: §4.1 · p. 29
  • Action fidelity is judged on faithfulness and precision across agent rollouts that preserve background stability and environment-level scene interventions.GPT-4o proposes feasible action sequences from an initial image, and a VLM-based judge scores the simulated rollouts.Source: §4.1 · p. 29
  • Long-horizon forecasting measures chunk-boundary motion smoothness and extended-rollout content and style consistency, with progressive penalties emphasizing late predictions.Transition Smoothness uses dense optical-flow velocity and acceleration; Simulation Consistency adapts metrics from WorldScore (Duan et al., 2025).Source: §4.1 · p. 29
  • Simulative reasoning and planning tests atomic robotic causal prediction and goal-directed manipulation through agent–world-model action-selection loops.Settings include Agibot scenarios and 47 Language Table tabletop test cases; OpenAI-o3 proposes candidate actions in the 15-scenario open-ended setting.Source: §4.1 · p. 29

4.2 Baseline Descriptions

  • Video-generation baselines include WAN 2.1-I2V-14B, WAN 2.2-I2V-14B, KLING, MiniMax-Hailuo, and Gen-3.Their descriptions emphasize image-to-video generation, controllability, visual fidelity, temporal coherence, or spatiotemporal control.Source: §4.2 · pp. 29–30
  • World-model baselines comprise Cosmos1-14B, Cosmos2-14B, and V-JEPA 2, spanning action-conditioned state prediction and future-latent prediction.Cosmos2 is described as improving long-horizon coherence and physics fidelity, while V-JEPA 2 predicts latent representations rather than pixels.Source: §4.2 · p. 29

5 Related Work

The section positions PAN among latent-dynamics models, domain-specific simulators, predictive representation methods, video generators, hybrid LLM–diffusion systems, and evolving evaluation benchmarks. It distinguishes PAN by combining action-conditioned temporal modeling with generative rollout for broad-domain simulation.

World models and latent dynamics

Interactive and domain-specific world models

  • Domain-specific models excel in target environments, while PAN targets broader language-action simulation, trading task-specific precision for domain generality.Examples span gaming, autonomous driving, robotics, and 3D scene generation; the latter typically emphasizes static structure over dynamic physics and action-conditioned rollout.Source: §5 · p. 30

Joint embedding predictive models

  • JEPA, V-JEPA (Bardes et al., 2024), V-JEPA 2 (Assran et al., 2025), and DINO-WM (Zhou et al., a) predict latent futures without reconstruction; PAN grounds transitions through generative reconstruction.The passage notes latent-only objectives may provide looser observation-level consistency and omit information needed for visual simulation, while reconstruction renders observable action consequences.Source: §5 · pp. 30–31

Video generation models

  • Video generators can produce realistic, longer outputs, but typically lack explicit action conditioning, sequential state evolution, and closed-loop rollout; PAN adapts Wan2.1-T2V-14B as its decoder.PAN adds a structured dynamics backbone and action-conditioned rollout to the decoder’s visual fidelity.Source: §5 · p. 31

Hybrid LLM–diffusion architectures

World model evaluation

  • WM-ABench, WorldScore (Duan et al., 2025), and Cosmos Bench (NVIDIA et al., 2025) assess distinct world-model capabilities, while WR-Arena (Gao et al., 2026) targets open-domain language-conditioned rollout.Atari (Bellemare et al., 2013) and DeepMind Control Suite (Tassa et al., 2018) remain benchmarks for closed-loop control with low-dimensional actions; WR-Arena evaluates action simulation, long-horizon forecasting, and simulative reasoning and planning.Source: §5 · p. 31

6 Additional Analysis

Across long rollouts, PAN degrades more gradually than standalone video generators, with stability arising from its training and generation framework and additional gains from its latent dynamics backbone. Ablations attribute smoother transitions and agent behavior to complementary components, while the training loss remains unsaturated at the end of the reported run.

6.1 Error Accumulation Over Long-Horizon Rollouts

  • PAN’s consistency falls from 87.0 at round 1 to 69.9 at round 9 yet leads from round 5; Wan-2.1 falls from 90.8 to 29.1 and Wan-2.2 from 89.8 to 30.8.Although Cosmos-2 starts at 93.0, it falls below PAN by round 5 and ends at 63.9. PAN and Wan-2.1 share the underlying generative capacity, but PAN’s stability is attributed to Causal Swin-DPM and the autoregressive latent dynamics backbone, not the decoder alone.Source: §6.1 · pp. 31–32, Figure S3 · p. 32

6.2 Ablation of the Causal Swin-DPM Denoising Mechanism

  • Causal Swin-DPM improves Transition Smoothness from 11.0 to 44.6 and Simulation Consistency from 36.2 to 60.9; Action Simulation Fidelity (Overall) improves from 45.3 to 56.8.The model attends to partially denoised context from the previous chunk instead of a single fully denoised final frame. The Action Simulation Fidelity improvement is associated with a world-modeling-oriented data pipeline that aligns actions with evolving scene state.Source: §6.2 · pp. 32–33

6.3 Ablation of the VLM-based Latent Dynamics Backbone

  • With the latent backbone, transition smoothness improves by 9.0 points (44.6 to 53.6), agent simulation by 4.3 points, and consistency from 60.9 to 64.1, while environment simulation is essentially unchanged (47.7 versus 47.0).The backbone supplies a persistent latent world-state representation that supports coherent transitions and language-grounded agent behavior; the decoder-only variant retains broadly similar environment simulation.Source: §6.3 · p. 33

6.4 Training Dynamics and Loss Trajectory

  • Approximately 2% loss reduction over Stage 2 indicates that training had not saturated by the end of the single pass.The trajectory is stable across both stages, and joint training resumes at the Stage 1 loss level; characterizing scaling requires runs at multiple data and compute budgets, left for future work.Source: Figure S4 · p. 34, §6.4 · pp. 34–35

7 Limitations and Future Work

PAN remains limited in precise low-level control and long-horizon stability, and inherits a video-generation backbone’s emphasis on perceptual realism over physical grounding. Future work should expand its action representations and develop simulation-focused training objectives, architectures, and data.

Language-level action interface

  • Natural-language, video-chunk actions support open-domain rollouts but are less suitable for precise low-level control, where action fidelity is lower.Future interfaces should support continuous controls, robot proprioception, and structured environment feedback.Source: §7 · p. 35

Long-horizon drift and failure modes

  • Long-horizon rollouts can show semantic drift, object inconsistency, visual artifacts, and reduced action controllability, especially in difficult scenes or out-of-distribution settings.The reported results indicate improved stability under evaluated settings, not a solution to indefinite-horizon simulation.Source: §7 · p. 35

Inherited limitations from the video generation backbone

  • PAN inherits Wan 2.1’s emphasis on perceptual realism over causal correctness and controllable state evolution, allowing plausible but physically incorrect interactions or hallucinated scene elements.The proposed next step is training from scratch with simulation-focused objectives, architectures, and data.Source: §7 · p. 35

Related papers

Loading 2511.09057v4…