PaperlayerView exact revision on arXiv

Source-linked AI summary

IntactWorld: Joint World Modeling with Intact Features

Boming Tan, Xiangdong Zhang, Yan Xia, Qi Zhu, Deyi Ji, Xue Yang, Shaofeng Zhang

arXiv:2610.11174v1cs.CV

TL;DR

Video generators can look realistic while struggling to model real-world logic, and existing approaches face feature-compression losses or high-dimensional optimization challenges. IntactWorld models uncompressed world features with clean-feature prediction and compact CLS-token guidance. It outperforms baselines and sets records on VBench and VBench 2.0, scoring 53.64 on VBench-2.0.

  • Problem

    Video generators can synthesize realistic visuals but often struggle to capture intrinsic real-world logic, while incorporating world knowledge is constrained by feature-compression losses and high-dimensional optimization.

    Source: §1 · p. 1

  • Method

    IntactWorld models uncompressed world features by predicting clean features x0 at intermediate layers, then uses compact CLS tokens for single-branch guidance.

    Source: §1 · pp. 1–2

  • Results

    IntactWorld outperforms baselines and establishes records on VBench and VBench 2.0, achieving 53.64 on VBench-2.0.

    Source: §1 · p. 2, §4.2 · p. 6

  • Takeaways & Limitations

    Condensed CLS tokens provide world-knowledge guidance with one additional feature branch alongside standard CFG, reducing inference costs.

    Source: §1 · pp. 1–2

    The target-selection rationale assumes structured clean data lie on a low-dimensional manifold, unlike noise and flow velocity.

    Source: §3.1 · p. 2

Abstract

from arXiv · show

While recent video generation models synthesize highly realistic visuals, they lack a genuine understanding of intrinsic real-world logic. Existing methods attempt to understand the world by internalizing diverse world knowledge, yet constrained by computational overhead or dimensionality alignment, their learning processes inevitably compress features, causing a severe loss of structural information. To address this, we propose \textbf{IntactWorld}, a \textbf{Joint World Modeling Architecture} utilizing uncompressed \textbf{Intact Features}. Since data naturally reside on a low-dimensional manifold within a high-dimensional space, predicting the flow velocity $v$ within this uncompressed high-dimensional space induces a severe manifold gap. To successfully eliminate this optimization bottleneck, our framework instead predicts the clean feature $x_0$ at intermediate layers. Furthermore, to mitigate the computational overhead of incorporating complete world knowledge, we introduce a \textit{Full-to-Compact Training Paradigm}. By replacing raw full features with highly refined CLS tokens, this paradigm enables efficient single-branch guidance, reducing spatial memory consumption by 11.4\% and cutting inference latency by 43.8\%. Extensive evaluations demonstrate the effectiveness of IntactWorld, outperforming established baselines by 2.46 points on the VBench 2.0 benchmark.

1 Introduction

Video generators can produce realistic sequences but often struggle to capture intrinsic real-world logic. IntactWorld uses uncompressed world features and clean-feature prediction, then compacts learned knowledge for efficient guidance; experiments report state-of-the-art results on VBench and VBench 2.0.

  • Compression erases spatial and motion details, while directly predicting velocity in uncompressed high-dimensional features creates an optimization bottleneck.Source: §1 · p. 1
  • IntactWorld uses uncompressed DINOv2 and VGGT features, predicting clean world features x0 at intermediate layers while retaining velocity prediction for VAE latents.The training loss remains in velocity space.Source: §1 · pp. 1–2
  • The Full-to-Compact paradigm uses condensed CLS tokens for single-branch guidance alongside standard CFG, reducing the inference overhead of incorporating world knowledge.Source: §1 · pp. 1–2
  • IntactWorld outperforms baselines and DreamWorld, establishing new records on VBench and VBench 2.0.Source: §1 · p. 2

2 Related Works

Related work traces video generation from latent diffusion and transformer architectures to world-modeling approaches that integrate knowledge, and research that emphasizes respecting data geometry.

  • Video generation progressed from latent diffusion and UNet-based systems to DiT backbones and large-scale foundation models, often paired with Flow Matching.Source: §2 · p. 2
  • World-modeling research includes latent prediction for robotic planning, action-conditioned video models, joint integration of multiple knowledge sources, and retrieval-augmented strategies.These principles have also been applied to autonomous driving.Source: §2 · p. 2
  • Manifold-related work spans clean-data prediction and methods addressing geometric interference, distillation, concept manifolds, and manifold-aware video generation.Together, these works emphasize respecting data geometry in generative modeling.Source: §2 · p. 2

3 Method

IntactWorld jointly models video latents and uncompressed world features, then abstracts those features into compact CLS tokens for efficient guidance. Its method covers the flow-matching setup, feature preprocessing, full-feature training, token training, and inference guidance.

3 Method

  • The method combines flow-matching preliminaries, feature preprocessing, full-feature joint modeling, compact-token training, and single-branch inference guidance.The section presents IntactWorld as a joint world-modeling architecture with intact features.Source: §3 · p. 2

3.1 Preliminaries

  • The preliminaries use Flow Matching to model a path from video latents to Gaussian noise, while clean features lie on a low-dimensional manifold unlike velocity targets.The method motivates predicting clean data for dense world features while retaining the flow-matching framework.Source: §3.1 · pp. 2–3

3.2 Preprocessing

  • Preprocessing represents optical flow as three-channel RGB input for video encoders, while feature alignment matches expert-feature resolutions and standardizes channels.A frozen 3D VAE encodes the motion representation into temporal latents.Source: §3.2 · pp. 3–4

3.3 Stage I: Full Feature Training

  • Stage I retains full DINOv2 and VGGT features, predicts clean world features while retaining velocity prediction for VAE latents, and computes the world loss in velocity space.The clean-feature target addresses the manifold gap; cosine decay reduces world guidance as training proceeds.Source: §3.3 · p. 4

3.4 Stage II: CLS Tokens Training

  • Stage II replaces dense expert features with compact CLS tokens and uses position-isolated RoPE so tokens are processed without spatial-temporal coordinate constraints.The CLS tokens internalize structural knowledge learned in Stage I and support a single additional guidance branch.Source: §3.4 · pp. 4–5

3.5 Compact Inner-Guidance

  • Compact Inner-Guidance uses unified world tokens to reduce denoising-step forward passes from five to three while preserving world dynamics.The tokens act as a holistic world prior, replacing separate modality branches.Source: §3.5 · p. 5

4 Experimental Results

IntactWorld achieves strong benchmark results and reduces inference cost relative to DreamWorld. Its ablations support full-feature training, CLS-token abstraction, clean-feature prediction, and intermediate prediction at layer 22.

4.1 Implementation Details

  • Experiments use a 30K WISA-80K subset with 81-frame, 480 × 832 clips; full features are extracted for 24K videos for Stage I and CLS tokens for 6K for Stage II.Stage I trains for 1,500 steps and Stage II for 750 additional steps, initialized from Stage I LoRA weights.Source: §4.1 · pp. 5–6

4.2 Quantitative Results

  • 81.45 overall on VBench is state of the art; on VideoPhy, IntactWorld scores highest in Semantic Adherence (55.2) and Physical Commonsense (27.6).The VBench results particularly highlight Semantic and Spatial dimensions, including Object Class and Spatial Relationship.Source: §4.2 · p. 6
  • 3.6 GB less GPU memory (11.4%) and 43.8% lower inference latency (237 to 133 seconds per video) versus DreamWorld with Compact Inner-Guidance.Figure 3 compares inference latency and VRAM usage.Source: §4.2 · p. 6, Figure 3 · p. 7

4.3 Qualitative Results

  • Across dynamic settings, IntactWorld preserves object structure and temporal coherence where Baseline and DreamWorld show deformation and artifacts during rapid movement or complex interaction.Figure 2 marks structural errors and artifacts with red boxes and reports superior visual quality with fewer artifacts for IntactWorld.Source: Figure 2 · p. 6, §4.3 · p. 7

4.4 Ablation Studies

  • Skipping Stage I full-feature training degrades video quality and semantic alignment, while merging raw features into three inference branches underperforms across metrics amid denoising interference.Table 4 evaluates full-feature training and CLS-token abstraction.Source: §4.4 · p. 7, Table 4 · p. 7
  • Replacing clean-feature x0 prediction with velocity v causes a performance drop; the authors report x0 supervision as a more reliable, deterministic signal for structural integrity.Table 4 compares prediction targets x0 and v.Source: §4.4 · p. 7, Table 4 · p. 7
  • Layer 22 yields the best generation quality in a 10K-data sweep across layers 16, 18, 20, 22, and 24, with quality declining at layer 24.The reported trend is improvement with deeper prediction through layer 22, followed by degradation at layer 24.Source: §4.4 · p. 7, Figure 4 · p. 7

5 Conclusion

IntactWorld models intact world features by predicting clean features at intermediate layers and computing the loss in velocity space to address the manifold gap. Its Full-to-Compact Training Paradigm and Compact Inner-Guidance use CLS tokens for efficient single-branch guidance, and experiments establish new records on VBench and VBench-2.0.

5 Conclusion

  • IntactWorld combines clean-feature prediction at intermediate layers with velocity-space loss, then uses compact CLS tokens and single-branch guidance to reduce inference overhead while setting new records on VBench and VBench-2.0.The conclusion describes a Full-to-Compact Training Paradigm and Compact Inner-Guidance mechanism.Source: §5 · p. 7

Related papers

Loading 2610.11174v1…