PaperlayerView exact revision on arXiv

Source-linked AI summary

Next Forcing: Causal World Modeling with Multi-Chunk Prediction

Gangwei Xu, Qihang Zhang, Jiaming Zhou, Xing Zhu, Yujun Shen, Xin Yang, Yinghao Xu

arXiv:2606.11187v1cs.CV

TL;DR

Autoregressive world models rely on myopic next-chunk supervision and slow iterative denoising, limiting training and inference efficiency. Next Forcing predicts multiple future chunks causally, achieving state-of-the-art RoboTwin results, 2.3× faster convergence, and accelerated inference.

  • Problem

    Teacher-forced next-chunk denoising provides myopic supervision, especially at high frame rates, where standard models converge more slowly with lower final accuracy.

  • Method

    Next Forcing adds lightweight causal MCP modules that predict next^1, next^2, and next^3 video chunks using fused intermediate features.

  • Results

    Next Forcing achieves 94.1/93.5% on RoboTwin Clean/Random, 2.3× faster convergence at 50 fps, 2× inference acceleration, and over 50% FVD reduction.

  • Takeaways & Limitations

    Next Forcing improves causal world-model training and inference across robot, physical-law, and general-video evaluations.

  • Takeaways & Limitations

    The MCP modules introduce extra training cost.

Abstract

from arXiv · show

Autoregressive video generation has emerged as a powerful paradigm for World Action Models (WAMs). However, existing approaches suffer from slow training convergence and limited converged accuracy, particularly at high frame rates, as the training supervision is confined to the current chunk without explicit signals about future dynamics; they also suffer from slow inference due to iterative video denoising. In this paper, we present Next Forcing, a multi-chunk prediction (MCP) framework for causal world modeling that enables faster training, higher accuracy, and accelerated inference. Inspired by multi-token prediction in large language models, Next Forcing introduces an MCP training objective that augments the main model with lightweight auxiliary MCP modules to simultaneously denoise video chunks at multiple future temporal horizons (next$^1$, next$^2$, next$^3$ chunks). These MCP modules form a causal chain across prediction depths, where intermediate features fused from multiple layers of the main model are leveraged to predict future dynamics, allowing near-future predictions to inform farther-future ones and providing dense multi-scale temporal supervision back to the main model. During training, the MCP modules significantly accelerate convergence and improve converged accuracy, especially at high frame rates: at 50 fps, Next Forcing achieves a 93.1% relative improvement over LingBot-VA at 5k training steps and 2.3x faster convergence, and establishes new state-of-the-art results on the RoboTwin benchmark (94.1/93.5% on Clean/Random). At inference, the MCP modules can be retained to predict the next video chunk in parallel with the current one, achieving 2x inference acceleration. Next Forcing also demonstrates significant improvements on PhyWorld, a benchmark evaluating adherence to physical laws in video generation, and over 50% FVD reduction on general video pretraining.

1 Introduction

Next Forcing addresses myopic supervision in autoregressive video world models by training chained auxiliary modules to predict multiple future chunks. It improves training convergence and accuracy while enabling parallel inference, with gains on RoboTwin, PhyWorld, and general video pretraining.

  • Training: 50 fps: Next Forcing achieves 2.3× faster convergence and higher converged accuracy than LingBot-VA.At 5k steps, it outperforms LingBot-VA by 29.7 points on Random and matches its 45k-step accuracy at 20k steps.
  • Method: Next Forcing replaces single-chunk supervision with multi-chunk prediction across next1, next2, and next3 horizons to learn longer-range temporal dynamics.Chained auxiliary MCP modules fuse intermediate main-model features, propagating supervision across prediction depths.
  • Benchmark results: 94.1/93.5% on Clean/Random: Next Forcing establishes new state-of-the-art results on RoboTwin.The evaluation covers multiple frame rates.
  • Generalization: Over 50% FVD reduction: Next Forcing improves general video pretraining beyond robot-specific data and achieves substantial gains on PhyWorld.The general-video result uses 3.5M in-house video clips.
  • Inference: 2× inference acceleration: retained MCP modules predict the next video chunk in parallel with the current one.The same modules that provide dense temporal supervision during training can accelerate generation at inference.

2 Related Work

Related work distinguishes WAMs from VLAs by their use of video prediction for robot policy learning, while autoregressive video generation has become central to action-conditioned world models. Prior training methods address exposure bias from teacher forcing by exposing models to noisy context during training.

  • World Action Models: WAMs predict future visual dynamics before decoding robot actions, unlike VLAs that directly map visual observations and language instructions to actions.This distinction defines WAMs as video-prediction-based approaches to robot policy learning.
  • Autoregressive Video Generation: Autoregressive video generation has become the core generation paradigm for action-conditioned world models and WAMs.The passage identifies autoregressive generation as the dominant paradigm in this setting.
  • Training Objectives: Teacher forcing trains on clean ground-truth context, creating exposure bias because inference uses self-generated context.Exposure bias is the resulting distribution gap between training and inference contexts.
  • Training Objectives: Diffusion Forcing mitigates exposure bias by independently sampling noise levels per frame, exposing the model to noisy context during training.This changes the training context from uniformly clean conditioning to frame-wise noisy conditioning.

3 Preliminaries

The paper formulates flow matching as velocity-field learning from noise to data and builds on LingBot-VA’s chunked autoregressive video-action framework. Its teacher-forcing setup matches closed-loop deployment but supervises only the current chunk, creating myopic supervision.

  • Flow Matching: Flow matching learns a velocity field that transports Gaussian-noise samples to the data distribution through linear interpolation.Given clean sample x0 and noise ϵ ∼ N(0, I), the noisy sample is xt = (1 − t)x0 + tϵ.
  • Flow Matching: The model trains a neural network vθ(xt, t, c) to predict the target velocity v∗ = ϵ − x0 conditioned on context c.At inference, an ODE solver integrates the velocity field from t = 1 to t = 0 to generate clean samples.
  • Autoregressive Video-Action Modeling: LingBot-VA encodes videos into latents with a pre-trained VAE and autoregressively denoises chunks of M frames at each step.The current noisy chunk is denoised using ground-truth data and a language instruction ℓ.
  • Autoregressive Video-Action Modeling: Teacher forcing aligns training with closed-loop deployment, but supervision confined to the current chunk produces a myopic supervision problem.Ground-truth observations replace generated frames after each action execution.

4 Next Forcing

Next Forcing extends teacher forcing with multi-chunk prediction, supervising future chunks through lightweight causal MCP modules and multi-layer main-model features. The modules can be discarded for zero-overhead deployment or retained for 2× inference acceleration using the same checkpoint.

  • MCP framework: Multi-chunk prediction supervises the current chunk and next^1, next^2, and next^3 chunks through three auxiliary modules arranged in a causal chain.The objective explicitly targets future dynamics, addressing myopic supervision and encouraging trajectory-level temporal reasoning.
  • MCP framework: Randomly sampled chunk lengths improve robustness across temporal scales, while independently noised shifted targets and future-offset RoPE provide depth-specific training signals.Chunk size M is sampled from {1, . . . , Mmax}; MCP timesteps use smcp > smain to bias modules toward higher noise levels and strengthen reliance on main-model representations.
  • Feature fusion: Multi-layer fusion collects hidden states from layers {4, 12, 20, 30}, enabling MCP gradients to supervise both early and late main-model representations.The fused features include noisy current and clean history latents, capturing denoising state and ground-truth context.
  • Joint video-action modeling: The unified Mixture-of-Transformers jointly predicts future visual dynamics and decodes actions via inverse dynamics, with improved video representations benefiting the action stream through cross-modal attention.Action decoding conditions on observations including the predicted future chunk.
  • Inference modes: 2× inference acceleration is achieved by retaining the depth-1 MCP module to generate the next chunk alongside the current chunk in one denoising trajectory.Alternatively, all MCP modules can be discarded for baseline-parity latency and memory, and both deployment modes use the same trained checkpoint without retraining.

5 Experiments · 5.1 Experimental Setup · 5.2 Main Results

Experiments evaluate Next Forcing on RoboTwin, PhyWorld, and general video pretraining, showing state-of-the-art manipulation success, faster convergence, improved physical consistency, and lower video-generation FVD than LingBot-VA. The method uses three lightweight MCP prediction depths with weighted auxiliary losses alongside the LingBot-VA framework.

  • 5.1 Experimental Setup: The evaluation covers RoboTwin’s 50 bimanual manipulation tasks under Clean and Random settings, reporting average task success rates.PhyWorld evaluates adherence to physical laws in generated videos.
  • 5.1 Experimental Setup: Next Forcing uses three MCP prediction depths—next1, next2, and next3—with three transformer blocks each and loss weights w1 = 0.5, w2 = 0.2, and w3 = 0.1.It is built on LingBot-VA with a 30-layer Wan2.2 Transformer backbone while keeping baseline settings identical.
  • 5.2.1 Comparison with State-of-the-Art: 94.1% and 93.5% are Next Forcing’s highest RoboTwin success rates on Clean and Random, respectively, surpassing all compared VLA and WAM methods.These results establish state-of-the-art performance on both settings.
  • 5.2.2 Training Convergence Analysis: At 12 fps, Next Forcing reaches 90.0% at 10k steps versus LingBot-VA’s roughly 20k steps and finishes at 94.1/93.5% versus 92.8/91.8% on Clean/Random.At 50 fps and 5k steps, it achieves 70.2/61.6% versus 45.5/31.9%, and at 50k steps reaches 91.8/90.5% versus 88.6/85.2%.
  • 5.2.2 Training Convergence Analysis: At high frame rates, multi-chunk prediction supplies harder future-chunk supervision because farther chunks differ visually, discouraging appearance-copying shortcuts.This forces the model to learn temporally aware physical dynamics, explaining the larger 50 fps gains.
  • 5.2.3 PhyWorld Benchmark: Next Forcing improves both FVD and Abnormal Ratio over LingBot-VA on PhyWorld, with larger gains in the out-of-template setting.Qualitative comparisons likewise show more physically consistent dynamics than the baseline.
  • 5.2.4 Generality on Video Pretraining: At 50k steps, Next Forcing reduces FVD by 58% on Test Set 1 (94 vs.225) and 52% on Test Set 2 (97 vs.204).It already surpasses LingBot-VA at 50k steps after only 10k steps on both test sets.

5.3 Ablation Studies

Ablations on RoboTwin Clean show that both baseline training choices and MCP architecture choices materially affect performance. Multi-layer fusion, smaller MCP timestep shifts, and lightweight modules improve supervision and coupling, while three transformer blocks are retained to reduce visual artifacts.

  • Evaluation setup: All ablations use the RoboTwin Clean subset and are trained and evaluated for 20k steps.The study covers both baseline and MCP module design choices.
  • Baseline ablation: 75.6% is achieved by the default baseline with s_main = 5 and noisy history augmentation, versus 69.8% without augmentation.Without noisy history, the model may shortcut denoising by copying clean context instead of learning dynamics.
  • MCP module ablation: 83.6% follows removal of multi-layer fusion, while setting s_mcp = 5 reduces performance to 83.2%.Multi-layer fusion enables deeper gradient propagation, and a higher MCP shift increases reliance on main-model representations.
  • MCP module ablation: 86.5% with one transformer block exceeds 85.0% with five blocks, but three blocks remain the default because they produce fewer visual artifacts.The artifact reduction matters for parallel chunk generation at inference; without weight initialization, performance is 83.8%.

5.4 Inference Acceleration

Retaining MCP modules during inference enables parallel prediction of the next video chunk, reducing video denoising cost while maintaining comparable accuracy across frame rates.

  • Inference Acceleration: MCP modules predict the next video chunk in parallel with the current one, significantly reducing video denoising cost.This inference strategy retains the MCP modules described in Section 4.6.
  • Inference Acceleration: 2x inference acceleration is achieved while maintaining comparable accuracy to the standard pipeline across all frame rates.

6 Conclusion

Next Forcing addresses myopic supervision in autoregressive video world models by training lightweight multi-chunk prediction modules to forecast multiple future chunks alongside the main model. This dense temporal supervision encourages learning long-range dynamics, while achieving state-of-the-art RoboTwin results and a reported 2.3× speed improvement.

  • Framework: Next Forcing addresses myopic supervision in autoregressive video world models through multi-chunk prediction.Lightweight MCP modules predict multiple future chunks alongside the main model.
  • Training objective: Dense temporal supervision forces the model to learn long-range dynamics rather than relying on appearance shortcuts.The supervision comes from predicting multiple future chunks during training.
  • Results: 94.1/93.5% on Clean/Random establishes new state-of-the-art results on RoboTwin.The passage also reports a 2.3× speed improvement, although its specific target is truncated.

A Attention Mask Details

Next Forcing uses a shared attention mask for the main model and MCP modules over noisy target tokens and clean context tokens. The mask enforces causal information flow while preventing current-chunk ground-truth leakage and clean-to-noisy attention.

  • Sequence structure: The shared sequence contains noisy tokens for the current chunk and clean tokens providing ground-truth context from previous chunks.Only video tokens are shown; action tokens are omitted for clarity.
  • Attention rules: Noisy tokens attend to causally preceding clean tokens, excluding the clean token at the same chunk index, and to noisy tokens within their chunk.This prevents information leakage from the current chunk’s ground truth while enabling within-chunk self-attention.
  • Attention rules: Clean tokens attend causally to clean tokens at the same or earlier chunk indices, while clean-to-noisy attention is prohibited.The clean-token pattern follows standard causal attention, and clean context tokens cannot attend to noisy tokens.
  • Mask sharing: The MCP modules reuse the main model’s mask because both process noisy target tokens with clean context tokens.The mask is constructed once per training step and shared across the main model and all MCP depths, reducing training overhead.

B Detailed Training Convergence Results

Next Forcing consistently outperforms LingBot-VA during RoboTwin training, with the largest gains appearing early and becoming substantially larger at 50 fps. The results attribute this faster improvement to multi-chunk prediction providing stronger supervision and preventing appearance shortcuts.

  • Training convergence: Next Forcing consistently outperforms LingBot-VA across RoboTwin training at both 12 fps and 50 fps.At 12 fps, its lead is approximately 10 points at 5k steps and remains 1–2 points at convergence; at 50 fps, the advantage is larger and persists through convergence.
  • Training convergence: 24.7/29.7 points: Next Forcing leads LingBot-VA at 5k steps at 50 fps on Clean/Random.The 50 fps gap is substantially larger than at 12 fps and persists through convergence.
  • Training convergence: The largest absolute improvement occurs during early training at 5k–10k steps.The paper links this early gain to multi-chunk prediction providing a stronger learning signal from the start by preventing appearance shortcuts.
  • Frame-rate dependence: MCP improvements are frame-rate dependent because appearance shortcuts are less severe at 12 fps than at higher frame rates.At 12 fps, adjacent chunks differ substantially in visual content, allowing the baseline to learn meaning more effectively.

C Timestep Shift Formulation

The timestep shift mechanism constructs a transformed noise schedule for both the main model and MCP modules, then samples training timesteps uniformly from that shifted schedule. The shift redistributes noise levels toward noisier regimes, using s_main = 5 and s_mcp = 10.

  • Shifted Timestep Schedule: Larger s pushes shifted noise levels toward higher values, concentrating training on noisier regimes; s_main = 5 and s_mcp = 10.The shift mechanism is used in both the main model and MCP modules.
  • Shifted Timestep Schedule: T = 1000 uniformly spaced base values are constructed over [σ_min, σ_max] = [0, 1] and transformed by shift parameter s.The transformed schedule is monotonic and maps [0, 1] to [0, 1].
  • Timestep Sampling: During training, id is sampled uniformly from {0, 1, ..., T − 1}, and the corresponding shifted noise level σ_tilde_id is retrieved from the precomputed schedule.Uniform index sampling over the shifted schedule induces a non-uniform distribution over noise levels.
  • Timestep Sampling: The noisy training sample is formed as x_tilde_σ = (1 − σ_tilde_id) x_0 + σ_tilde_id ε, with ε sampled from N(0, I).The shifted noise level controls the interpolation between the clean sample and Gaussian noise.
Loading 2606.11187v1…