DyMD: Preserving Interaction Dynamics through Distribution Matching Distillation in Few-Step Video World Models

• R-Bench task adherence rises by 9.6 percentage points and PAI-Bench-G Domain score by 5.1 points over Base DMD; WorldArena mean success is 34% versus 16%.Both students are 1.3B four-step models, and visual quality remains comparable.Source: Figure 1 · p. 1, §1 · p. 2
• Video world models support embodied prediction and learning, but large diffusion models require costly iterative sampling, motivating few-step distillation.DMD uses the score difference between a frozen teacher and an online fake-score model to train a few-step student.Source: §1 · p. 1
• Weak re-noising keeps the teacher posterior near motion-deficient rollouts, whereas stronger re-noising can restore robot–object motion but introduce artifacts.Typical schedules use the same timestep distribution regardless of each rollout’s interaction fidelity.Source: §1 · p. 2, Figure 2 · p. 2
• Stronger-motion rollouts tend to have higher critic flow-matching loss, indicating greater fitting difficulty that can hinder learning interaction dynamics.The association is reported at fixed noise levels using frozen rollouts.Source: §1 · p. 2, Figure 3 · p. 2
• DyMD couples rollout-adaptive teacher supervision with difficulty-aware critic fitting through temporal affinity–conditioned re-noising and dynamics-guided fake-score tracking.Low-affinity rollouts receive more teacher-prior sampling, while predicted-hard rollouts receive larger critic-loss weights without extra critic updates.Source: §1 · p. 2

Video World Models

• Video world models extend visual-generation priors to embodied prediction, where object persistence, contact consistency, and coordinated robot–object motion matter.VideoJAM models appearance and motion [5], ABot-PhysWorld uses physics-aware preference optimization [7], and PhysisForcing adds region-focused physics supervision [36].Source: §2 · pp. 2–3

Diffusion Model Distillation

• Few-step distillation transfers pretrained models through trajectory consistency [21] or distribution-level supervision, including adversarial learning and score-based distribution matching.Extensions address reward guidance [16] [20], temporal fidelity, reconstruction-aligned updates [31], and relational structure across samples and frames [37].Source: §2 · p. 3

Timestep Sampling and Fake-Score Tracking

• Prior approaches improve timestep allocation or fake-score accuracy through loss-aware sampling, SNR weighting, phased matching, and consistency or update strategies [14].DyMD instead adapts each rollout’s re-noise schedule using temporal affinity and reweights critic samples using predicted noise-relative fitting difficulty from latent temporal dynamics.Source: §2 · p. 3

3. Method

• DyMD combines rollout-adaptive teacher supervision with difficulty-aware critic fitting; both generator and critic share timestep sampling, and auxiliary modules are training-only.The critic weights its flow-matching losses without changing their targets.Source: §3 · p. 3, Figure 4 · p. 4

3.1. Preliminaries: DMD and Its Challenges

• DMD’s teacher guidance can remain localized around motion-deficient rollouts, while critic lag passes score-estimation errors into the generator update.Stronger re-noising can relax the teacher’s local constraint but may weaken appearance evidence; the critic challenge motivates allocating limited fitting effort to harder rollouts.Source: §3.1 · pp. 3–4

3.2. Teacher-Velocity Turning Prior Schedule

• The method uses squared mean velocity turning as a geometric proxy for posterior variation, with the proposition relying on regularity and boundedness assumptions.Large turning marks pronounced denoising-direction reorganization.Source: §3.2 · p. 4

3.3. Temporal Affinity–Conditioned Re-noise Sampling

• Temporal affinity compares rollout and reference-video changes in feature space; low-affinity rollouts receive more teacher-prior sampling, while well-matched ones retain base-schedule coverage.Generator and critic each use the timestep distribution computed from their respective rollouts.Source: §3.3 · p. 5

3.4. Dynamics-Guided Fake-Score Tracking

• Because stronger latent motion is associated with higher flow-matching loss, a noise-conditioned predictor estimates fitting difficulty from latent temporal dynamics and upweights predicted-hard rollouts.The weights emphasize harder samples within a fixed critic-update budget and are detached from the critic objective.Source: §3.4 · pp. 5–6

4.1. Experimental Setup

• The PF-Wan 14B teacher is distilled into a bidirectional 1.3B student generating 81-frame videos in four evaluations, using 32,897 curated clips.Evaluation includes R-Bench, PAI-Bench-G, and EZS-Bench; all models use the same initial images, prompts, and random seeds.Source: §4.1 · pp. 6–7

4.2. Main Results

• At four NFE, DyMD raises R-Bench TAC to 44.2 (+9.6 points) and PAI/EZS Domain to 79.5/76.2 (+5.1/+2.9 points) over Base DMD.Base DMD uses 4 NFE, versus 80 for the teacher and 100 for the undistilled student.Source: §4.2 · p. 7

4.3. Representation Analysis for the Generator and Critic

• V-JEPA temporal affinity ranks target-relative motion more closely than VAE (Spearman 0.374 vs. 0.162), while VAE descriptors better track noise-relative FM difficulty.Figure 7 compares temporal-affinity rankings with trajectory agreement and MLP predictions with log-FM loss across noise levels.Source: §4.3 · p. 7, Figure 7 · p. 7

4.4. Ablation Study

• Adaptive sampling and critic tracking complement one another, while schedule and weighting ablations favor rollout-specific sampling and moderate hard-rollout emphasis.Sampling alone raises R-Bench TAC/PAI Domain from 34.6/74.4 to 43.5/78.5; adding tracking reaches 44.2/79.5. The teacher prior improves over Shift-25, while affinity conditioning gives the best reported R-Bench TAC and Domain scores; excessive β = 1.0 reduces scores.Source: §4.4 · pp. 7–8, Table 2 · p. 7, Table 3 · p. 8

4.5. Qualitative Results

• Qualitative cases contrast Base DMD’s incomplete container lift and menu handover with DyMD’s completed interactions, while preserving scene appearance.Source: §4.5 · p. 8

4.6. Downstream Action Planning

• As frozen 1.3B backbones, DyMD raises WorldArena mean success from 16.0% to 34.0% across Adjust Bottle and Click Bell.Task-specific VPP heads are trained from 50 demonstrations per task and evaluated over 100 closed-loop episodes per task.Source: §4.6 · p. 8, Table 5 · p. 8

• The four-step 1.3B student improves R-Bench TAC and PAI-Bench-G and EZS-Bench Domain scores over Base DMD, and raises WorldArena mean success from 16% to 34%.Source: §5 · p. 8
• Temporal affinity–conditioned re-noise sampling adapts the teacher-velocity-turning prior and base schedule to each rollout’s interaction fidelity; dynamics-guided tracking reweights critic loss by predicted fitting difficulty.Source: §5 · p. 8
• The turning profile is teacher- and schedule-specific and must be re-estimated for a new teacher.Source: §5 · p. 8

• Caption deduplication removes exact duplicates, then uses a greedy cosine-similarity threshold of 0.95 to retain 448,528 representatives.Processing captions by descending frequency avoids merging through chains of intermediate matches.Source: A. Training-Set Curation · p. 11
• Image–text filtering retains the top 50% of clips by first-frame and caption similarity, leaving 224,264 clips.This stage filters visual–textual mismatches not detected by caption-only deduplication.Source: A. Training-Set Curation · p. 11
• CoTracker3 motion and visibility filtering retains clips with at least 50 selected tracks and mean visibility of at least 0.6, yielding 32,897 clips.Source: A. Training-Set Curation · p. 11

B.1. Image-to-video conditioning

• Image conditioning encodes the input image as the first frame, with subsequent frames set to zero and a mask marking only the first latent.The student performs four bidirectional denoising evaluations and generates 81 frames.Source: §B.1 · p. 11

B.2. Video features for temporal affinity

• V-JEPA features are extracted from 16 uniformly sampled RGB frames, pooled within a fixed 3 × 3 full-frame grid, and normalized.Paired target features are cached as 8 × 9 × 1024 fp16 features.Source: §B.2 · p. 11

B.3. Teacher prior schedule and re-noise sampling

• The base schedule samples raw model time uniformly before flow-shift transformation; re-noising uses T = [0.020, 0.999), where shifts above 1 move times toward higher noise.Shift-25 approximately matches the teacher prior’s probability mass at SNR < 1.Source: §B.3 · p. 11

C. Geometry and Posterior Interpretation of Teacher-Velocity Turning

• The analysis connects DMD score differences to posterior means and relates teacher-velocity turning to posterior variation.It analyzes high-SNR mode locking in a two-mode model.Source: C. Geometry and Posterior… · p. 11

C.1. Score difference and posterior means

• For the same noisy state and context, the exact score difference depends on the student and teacher clean posterior means; high SNR can concentrate the teacher posterior near a motion-deficient student rollout.Source: §C.1 · pp. 11–12

C.2. Mode-Locking Analysis

• Proposition 2 models the teacher distribution as static and moving modes with positive prior weights that sum to one.The re-noised student sample is collapsed at the static mode.Source: §C.2 · p. 12
• The analysis defines mode separation as the difference between the moving and static modes and evaluates SNR under the re-noising process.The passage introduces the setup but does not supply the full posterior-odds expression.Source: §C.2 · p. 12
• The expected log-odds of the moving mode decrease with SNR and squared mode separation, favoring the static mode at high SNR.The proof reduces the posterior odds to prior odds times a ratio of Gaussian likelihoods.Source: §C.2 · p. 12

C.3. Posterior Variation and Proof of Proposition 1

• The operator H excludes the observed block and subtracts the temporal mean over future blocks.The setup applies H to latent temporal blocks when defining the posterior-path quantities.Source: §C.3 · p. 12
• The pointwise bound relates posterior-mean variation to the turning statistic for the linear noising process.The proof differentiates the posterior mean along a characteristic and then bounds the resulting variation.Source: §C.3 · p. 12
• The average bound requires uniformly nonzero noise-scaled centered velocity and bounded projected posterior covariance over the trajectories and noise interval.Under these assumptions, the proof takes expectations and applies Jensen’s inequality to establish Eq. (9).Source: §C.3 · p. 13
• The empirical turning statistic is motivated, not proven exact: analysis assumes the exact conditional velocity and continuous trajectories, unlike the discrete CFG-guided trajectories used in practice.The practical statistic is estimated from numerical trajectories of the guided teacher field.Source: §C.3 · p. 13

D.1. Representations for temporal affinity

• V-JEPA affinity ranks agree with CoTracker trajectory agreement more than VAE affinity, with Spearman correlations of 0.374 and 0.162, respectively.The comparison uses 32 frozen student rollouts and paired targets, with matched temporal lags and weights.Source: §D.1 · p. 13

D.2. Motion magnitude and critic fitting difficulty

• VAE temporal descriptors predict log-FM loss better than V-JEPA features, with Spearman correlations of 0.460 ± 0.048 and −0.254 ± 0.125, respectively.Hardest-quartile recalls are 36.7 ± 5.9% and 12.5 ± 5.7%, respectively, against 25% chance.Source: §D.2 · p. 13

D.3. Difficulty ranking by the trained predictor

• The trained difficulty predictor correlates with measured critic FM loss at 0.758–0.952 on 32 fresh rollouts.Correlations are computed at each noise level using loss averaged over three independent re-noising draws.Source: §D.3 · p. 13, Table S2 · p. 13

E.1. Dynamics-guided fake-score tracking

• The tracking predictor uses latent temporal-difference summaries together with a noise-level embedding to estimate rollout fitting difficulty.The temporal descriptor summarizes RMS latent differences at lags 1, 2, 4, and 8, plus displacement relative to the first block.Source: §E.1 · p. 13

F. Benchmark Protocols and Metrics

• R-Bench evaluates 400 image–prompt pairs across four robot embodiments and averages normalized task-completion ratings equally across embodiments.GPT-5 rates each video from six uniformly sampled frames.Source: F. Benchmark Protocols and Metrics · p. 14
• PAI-Bench-G evaluates 913 binary questions over 174 robot conditions, with Domain as per-video accuracy and Space, Physics, and Time as per-axis question accuracies.PAI Quality is the unweighted mean of eight official dimensions, including subject consistency, motion smoothness, and image-to-video fidelity.Source: F. Benchmark Protocols and Metrics · p. 14, Table S4 · p. 14
• The video-native versus framewise affinity comparison uses V-JEPA 2.1-L and DINOv2-L under the same training budget.The table caption identifies the better result by bold blue shading but does not provide values in the supplied passage.Source: Table S5 · p. 14
• EZS-Bench covers 196 unseen robot–task–scene conditions and 2,272 binary questions, using the PAI-Bench-G per-video and per-axis aggregation.Action-centric prompts are compressed from original descriptions while original questions and checklists remain unchanged for scoring.Source: F. Benchmark Protocols and Metrics · p. 14

F.1. Downstream action-planning protocol

• The downstream protocol extracts frozen student features in a single partial forward pass, trains a separate VPP head per backbone and task, and evaluates closed-loop action chunks in RoboTwin 2.0.VPP predicts 45-action chunks with ten action-denoising steps and is evaluated over 100 episodes per task; qualitative comparisons show clearer object-directed motion for DyMD than Base DMD.Source: §F.1 · pp. 14–15, Figure S1 · p. 15

G.1. Video-native versus framewise temporal-affinity features

• V-JEPA improves TAC, PAI Domain, Quality, and EZS Domain over DINOv2 by 1.7, 1.8, 0.3, and 0.1 points, respectively.The comparison uses framewise DINOv2 features built from 16 sampled frames and eight temporal blocks.Source: §G.1 · p. 15

G.2. Teacher-prior probability floor

• The study uses α = 0.3 as a moderate floor because α = 0.1 provides insufficient teacher-prior supervision, while α = 0.5 yields comparable performance.At α = 0.5, TAC rises by 0.3 points and both Domain scores decrease by 0.3 points relative to α = 0.3.Source: §G.2 · p. 15
• With tracking disabled, raising α from 0.1 to 0.3 improves TAC, PAI Domain, and EZS Domain by 4.3, 1.1, and 1.9 points, respectively.Table S6 varies only α; the α = 0.3 row repeats the sampling-only result from Table 2.Source: Table S6 · p. 15, §G.2 · p. 15