S2PD: Serial-to-Parallel Diffusion for Physically and Logically Consistent Video Generation

1 INTRODUCTION

• Bidirectional models can assign locally plausible but mutually incompatible events early, then sharpen them without reconciling the resulting invalid transitions.The paper relates this hypothesis to video’s long chains of causally dependent events and the Serial Scaling Hypothesis.Source: §1 · p. 1
• On Rubik’s Cube real video, adapted models generate realistic hand–cube interactions while maintaining a plausible cube state over time.The paper also reports that serial methods outperform parallel generation on games, simulations, and more complex datasets.Source: §1 · p. 1, Figure 1 · p. 2
• S2PD preserves serial generation’s physical plausibility and logical consistency while improving sampling efficiency and temporal stability.It confines autoregressive diffusion to high noise, then refines the full sequence in parallel at low noise.Source: §1 · p. 2
• Bidirectional diffusion violates physical and symbolic rules despite effectively unlimited procedurally generated data, while autoregressive methods substantially reduce these errors.The paper reports this across games and physical simulations.Source: §1 · p. 2

Autoregressive generation

• Prior autoregressive video methods address training–inference mismatch or chunk-generation speed, while this work focuses on how spatio-temporal autoregression affects logical consistency.Self Forcing trains on generated video; Block Diffusion accelerates language modeling by generating chunks simultaneously.Source: §2 · pp. 2–3
• Block-causal diffusion generates chunks simultaneously; this work explores sub-frame block sizes to study how spatio-temporal autoregression affects logical consistency.Flex-Forcing uses block sizes larger than one frame as an acceleration technique with efficiency–quality trade-offs.Source: §2 · p. 3

World modeling

• World models are naturally autoregressive for real-time action input; this work studies consistency without temporally dense action conditioning and suggests world models may benefit from parallel refinement.The paper frames serial computation as relevant to the physics consistency demonstrated by MIRA in Rocket League.Source: §2 · p. 3
• Code-based scene reconstruction represents scenes as executable programs, whereas this work uses autoregressive video generation without human-labeled expert trajectories.The related-work passage also notes that VLMs supply serial computation through text.Source: §2 · p. 3

Evaluating physical consistency

• Real-world physics benchmarks make failure causes difficult to isolate, so the paper first studies games and simulations with quantitative feedback on rule violations and physical error.The cited benchmarks remain unsaturated while requiring real-world generalization.Source: §2 · p. 3

3.1 BACKGROUND

• The video is patchified into tubelets spanning consecutive frames, ordered into blocks whose tokens share a training noise level and are denoised together at inference.Smaller temporal patch sizes increase token count and training and inference time; a block spanning the full sequence is equivalent to bidirectional diffusion.Source: §3.1 · p. 3
• S2PD uses block-causal attention and separate context and target copies, with each target block attending to itself and preceding context blocks.Unlike bidirectional diffusion’s single video-wide noise level, Diffusion Forcing uses independent block noise levels; block-causal diffusion computes loss only on target blocks.Source: §3.1 · p. 4

3.2 SAMPLING

• Serial denoising generates blocks at high noise, then treats the full video as one block for parallel refinement from transition noise level τ to clean output.The default uses ten denoising steps and τ = 0.6, allocating four steps to serial denoising and six to parallel refinement; parallel batching is faster than serial denoising.Source: §3.2 · pp. 4–5

3.3 TRAINING

• Training mixes multiple block sizes and varies context noise so sampling can select block size and transition noise level at inference.S2PD otherwise uses the same token sequence, attention pattern, and noise levels as block-causal diffusion; the method is implemented in pixel-space DiT-B and pretrained latent-space Wan-5B-LoRA.Source: §3.3 · p. 5

4.1 DATASET COLLECTION

• The collection includes discrete datasets—Chess, 2048, 15 Puzzle, Tetris, Snake, and Rubik’s Cube 3D—continuous physical simulations, datasets of complex simulations, and real Rubik’s Cube footage.The continuous simulations comprise Double Pendulum, threebody orbital dynamics, 2D and 3D colliding balls, and Pong; Rubik’s Cube Real contains 30 minutes of scrambling-and-solving footage.Source: §4.1 · pp. 5–6

4.2 VIDEO PARSING AND EVALUATION

• For simple datasets, videos are parsed into symbolic states and checked against game rules or physical dynamics, with invalid transitions or cumulative dynamics error counted per rollout.The evaluation uses template matching for fixed-grid games, and object tracking plus fitted simulation parameters for continuous data.Source: §4.2 · p. 6
• For complex videos that cannot be parsed reliably, evaluation falls back to distributional metrics based on Fréchet distance.CD-FVD is the primary distributional metric because it is described as more sensitive to temporal distortions than FVD.Source: §4 · p. 5, §4.2 · p. 6
• Across datasets, serial methods outperform the bidirectional baseline and perform similarly to one another; on Conway, the baseline never learns valid transitions while ours rapidly learns the rules.Our method produces almost no Conway errors and multiple valid continuations from a single image.Source: §4.2 · p. 6, Figure 4 · p. 6, Figure 5 · p. 6

5 RESULTS

• Serial methods substantially outperform the bidirectional baseline across datasets and perform similarly to each other; matched serial Wan-5B-LoRA variants also improve complex-video coherence.On MPMWorlds, the standard bidirectional configuration produces implausible material deformations, while serial counterparts yield more reasonable continuations and perform better on Rubik’s Cube Real.Source: §5 · pp. 7–8
• With the same block size, S2PD achieves the lowest CD-FVD at twice the speed of other serial methods.For S2PD, τ = 0.6 is more than twice as fast as τ = 0.0 across all block sizes.Source: Table 3 · p. 8, §5 · p. 8
• S2PD’s parallel refinement smooths flicker artifacts and achieves the lowest CD-FVD on all four video datasets.Spatiotemporal slices show clean vertical lines for S2PD and ground truth, versus subtle patchy artifacts for BC and cDF.Source: §5 · p. 8, Figure 7 · p. 8

5.1 ABLATIONS

• τ = 0.6 scores similarly to τ = 0.2 while sampling significantly faster; τ = 0.0 is not recommended because it is highly susceptible to autoregressive drift.The transition noise level controls how much denoising occurs serially before parallel refinement begins.Source: §5.1 · p. 8, Table 4 · p. 9
• Smaller blocks reduce errors more on discrete than continuous datasets, while training for a specific block size improves that size’s performance at the cost of flexibility.Continuous datasets appear less sensitive to block-size choice.Source: §5.1 · p. 9, Table 5 · p. 9
• Zigzag and Hilbert token orders offer no consistent advantage over raster order, which the authors use by default.The token order determines the spatial grouping of tokens within small blocks.Source: §5.1 · p. 9, Table 6 · p. 9

6 CONCLUSION

• Across games and physical simulations, scaling training data does not enable bidirectional diffusion to model complex video.Serial methods outperform parallel baselines across many datasets and two architectures with matched model sizes and training budgets.Source: §6 · p. 9
• S2PD establishes video structure through autoregressive denoising at high noise levels, then refines details in parallel.This preserves physical and logical consistency while improving sampling efficiency and temporal stability over other serial methods.Source: §6 · p. 9
• The authors report using OpenAI Codex and Anthropic Claude for implementation, debugging, literature organization, experiment planning, and manuscript editing.The authors reviewed the AI-assisted work and take responsibility for the submission.Source: AI USE STATEMENT · p. 10

A IMPLEMENTATION DETAILS

• DiT-B uses pixel-space diffusion with x-prediction, while Wan-5B-LoRA uses latent-space diffusion with v-prediction; both use v-loss.Both architectures condition on text encoded by Wan’s frozen UMT5-XXL encoder, projected to transformer width and supplied through cross-attention.Source: A IMPLEMENTATION DETAILS · p. 13
• Training runs for 100 epochs of 1,000 optimizer steps, with batch sizes of 32 or 64 depending on method.Finite-dataset samples are repeated to fill epochs; EMA checkpoints are evaluated on validation splits, and simulated datasets use validation seeds distinct from training seeds.Source: A IMPLEMENTATION DETAILS · p. 13
• Training DiT-B takes about 12 hours on four H100 GPUs, while Wan-5B-LoRA takes about 48 hours on sixteen H100 GPUs.Each dataset uses a fixed prompt except MPMWorlds, which uses dataset-provided scene descriptions.Source: A IMPLEMENTATION DETAILS · p. 13

B DATASET DETAILS

• The transition residual measures the distance between an observed state and the nearest permitted next state.For consecutive parsed states, the permitted next states are defined from the preceding state.Source: B DATASET DETAILS · p. 13
• Discrete-dataset transitions are invalid if a state breaks dataset rules or no permitted move connects the states; the metric is invalid transitions per rollout.Persistent state errors are counted on every affected transition, and gameplay states require a move.Source: B DATASET DETAILS · p. 13
• Continuous-dataset dynamics error per rollout sums frame-level residuals averaged over source objects, using 1−IoU between predicted and observed regions.Predictions extrapolate from past frames using simulator dynamics, with a small positional tolerance; a missing expected object receives a full residual for each missing frame.Source: B DATASET DETAILS · p. 14

B.1 PER-DATASET DETAILS

• Conway requires each grid to follow from the previous grid under Conway’s rules, while Chess checks board validity, alternating turns, legal moves, and end-game modals.Chess checks include one king per side, pawn-rank restrictions, and whether a win modal follows visible checkmate.Source: §B.1 · p. 14
• 2048 requires a changing horizontal or vertical slide followed by a new 2 or 4 tile; Fifteen Puzzle requires every tile and one blank exactly once and legal adjacent slides.Source: §B.1 · p. 14
• Tetris checks vertical piece drops and row clearing, while Snake checks a connected chain, legal head and tail movement, food growth, and valid end-game conditions.Tetris allows any rotation of the dropped piece and any preview piece after a drop.Source: §B.1 · p. 14
• Rubik’s Cube 3D accepts transitions matching visible facelets under a tested camera orientation and legal quarter turn.The metric does not enforce color counts or solvability, and checks local motion rather than persistent identity through occlusion.Source: §B.1 · p. 14
• Colliding Balls 3D checks motion in table coordinates, while Colliding Balls 2D tracks balls by color and applies contact-law and overlap checks.The 3D rules accommodate plausible occlusion and border arrivals or departures; the 2D rules allow between-frame collisions and penalize unmatched arrivals or departures.Source: §B.1 · p. 14
• Double Pendulum motion is predicted from fitted rod lengths and mass ratio; Three Body motion uses fitted gravity and forces from visible bodies.The pendulum check penalizes unexplained missing bobs, while Three Body allows absences consistent with recent departure or occlusion.Source: §B.1 · p. 14
• Pong checks ball motion through wall and paddle collisions, requires paddles to stay on their rails and inside the court, and permits a serve-position return after a goal.Other missing or extra objects are penalized on affected frames.Source: §B.1 · p. 15

C.1 CONTEXT NOISE LEVEL

• With block size 64, S2PD scores lowest across methods at τ=0.6, while BC and cDF perform best at tc=0.2.Table 8 reports CD-FVD scores and varies the transition noise level for S2PD and re-noising for BC and cDF.Source: §C.1 · p. 15, Table 8 · p. 15

C.2 FVD

• S2PD has the lowest FVD on both Kubric datasets and MPMWorlds, while cDF and BC outperform it on Rubik’s Cube Real.Table 9 compares FVD for Wan-5B-LoRA at block size 64.Source: §C.2 · p. 15, Table 9 · p. 15

C.3 MPMWORLDS

• Table 10 reports unofficial MPMWorlds metrics reimplemented based on the paper.Source: Table 10 · p. 15
• The authors call the MPMWorlds metrics useless when scene conditions unavailable from the initial image significantly affect how it evolves.They identify strong code conditioning as an interesting direction for future work.Source: §C.3 · p. 16

• Figure 8 compares Bidirectional (top) and Ours (bottom) across seven discrete games, showing selected settled states at steps 0–8.Each step spans four frames at 24 frames per second; the displayed steps cover the first 1.33 seconds of each sample.Source: D ADDITIONAL QUALITATIVE RESULTS · p. 16, Figure 8 · p. 17
• Figure 9 shows Bidirectional above Ours for five continuous datasets, with one frame per second shown for the first eight seconds.The first frame in each row is the conditioning image.Source: D ADDITIONAL QUALITATIVE RESULTS · p. 16, Figure 9 · p. 18
• Figure 10 uses the same top-row Bidirectional and bottom-row Ours layout for four video datasets, showing selected frames from ten-second samples.Source: Figure 10 · p. 19