PaperlayerView exact revision on arXiv

Source-linked AI summary

ACWM-RM: A Reference-Free Reward Model from Complementary Mismatch Preferences

Shibo Hong, Siyu Ren, FengJiao Chen, Zhongyuan Peng, Wei Shi, Yixin Cao

arXiv:2610.32222v1cs.CV

TL;DR

Existing rewards for action-conditioned world models assess visual quality or action-following fidelity, but do not jointly provide reference-free supervision, and IDMs may be unreliable on generated videos. The paper introduces AVM-Eval and ACWM-RM, which combines complementary mismatch preferences with dynamics pretraining. The evaluation finds limited IDM reliability on generated videos and better VQ and AF outcomes from ACWM-RM in downstream RL.

  • Problem

    Existing metric-based rewards assess VQ but require ground-truth videos and do not directly assess AF, while IDM rewards assess AF without references but do not directly assess VQ and may be unreliable on generated videos.

    Source: §1 · p. 1

  • Method

    The paper constructs AVM-Eval and trains the reference-free ACWM-RM with complementary action- and video-mismatch preferences after pretraining an action-conditioned predictor on real videos.

    Source: §1 · p. 2

  • Results

    ACWM-RM outperforms alternative reward signals in VQ and AF when used to fine-tune DreamDojo-2B, while maintaining physical-consistency scores close to the pre-RL model.

    Source: §1 · p. 2, §6 · p. 9

  • Takeaways & Limitations

    Accurate AF assessment on real videos does not ensure reliable assessment on generated videos, so AVM-Eval evaluates IDM reliability across both settings.

    Source: §1 · p. 2, §6 · p. 9

    Downstream RL improvements have been demonstrated only for DreamDojo-2B on AgiBotWorld, and ACWM-RM does not explicitly supervise physical consistency.

    Source: §6 · p. 9

Abstract

from arXiv · show

Reinforcement learning has gained attention for improving action-conditioned world models (ACWMs), as it enables reward-driven optimization of visual quality (VQ) and action-following fidelity (AF) beyond flow-matching optimization. Realizing this potential requires reward signals that reliably rank videos by both VQ and AF. However, existing rewards capture only one dimension: metric-based rewards assess VQ but require ground-truth videos and do not directly assess AF, while inverse dynamics model (IDM) rewards assess AF without references but do not guarantee VQ. Moreover, IDMs, typically trained on real videos, may provide unreliable rewards for generated videos containing visual artifacts and physical inconsistencies. In this work, we introduce the Action and Video Mismatch Benchmark (AVM-Eval), comprising 67.2k pairs that evaluate how reliably IDMs assess AF on real and generated videos. We then propose ACWM-RM, a reward model that jointly assesses VQ and AF without ground-truth videos during reward computation. To train it, we first construct the Action and Video Mismatch Preference Dataset (AVM-Pref) with 506.3k pairs providing complementary supervision: action-mismatch pairs supervise AF, while video-mismatch pairs supervise VQ and expose the model to generated videos. We then pretrain an action-conditioned predictor on real videos to learn forward dynamics before jointly training it with a patch-weighted score head on AVM-Pref to transfer these dynamics to AF assessment on generated videos. Evaluation on AVM-Eval reveals the limited reliability of IDMs on generated videos, while downstream RL experiments show that ACWM-RM outperforms reward baselines in both criteria.

1 INTRODUCTION

Action-conditioned world models need reward signals that jointly rank visual quality and action-following fidelity, but existing rewards cover only one criterion or require reference videos. The paper introduces AVM-Eval and ACWM-RM, then reports that IDMs transfer unreliably to generated videos while ACWM-RM improves both criteria in RL fine-tuning.

1 INTRODUCTION

  • Metric-based rewards assess VQ but require ground-truth videos and do not directly assess AF, whereas IDM rewards assess AF without references but do not directly assess VQ.IDMs trained on real videos may be unreliable on generated videos with artifacts and physical inconsistencies.Source: §1 · p. 1
  • ACWM-RM jointly assesses VQ and AF without ground-truth videos at reward time, using AVM-Pref’s complementary action-mismatch and video-mismatch supervision.The dataset contains 506.3k pairs; the model first pretrains an action-conditioned predictor on real videos, then trains it with a patch-weighted score head on AVM-Pref.Source: §1 · p. 2
  • 86.1% accuracy on real-video action-mismatch pairs versus 70.0% on generated-video video-mismatch pairs for the best-performing IDM shows that real-video accuracy does not ensure reliable generated-video AF assessment.The evaluation covers four architecturally distinct IDMs.Source: §1 · p. 2
  • On held-out tasks, ACWM-RM yields the best VQ and AF among the compared rewards while maintaining VLM-assessed physical-consistency scores close to the pre-RL model.The paper situates this approach in RL for optimizing action-conditioned world models beyond flow-matching training.Source: §1 · pp. 1–2

2 RELATED WORK

Prior action-conditioned world models learn video prediction and support embodied simulation, evaluation, and planning. Existing rewards for world models assess visual quality or alignment, but the cited approaches do not jointly assess visual quality and correspondence with numerical actions without references.

2.1 ACTION-CONDITIONED WORLD MODELS

2.2 REWARD MODELING FOR WORLD MODELS

3 PRELIMINARIES

The preliminaries describe reward-based optimization of action-conditioned world models with Flow-GRPO. It scores generated videos using metric-based or IDM-based rewards, then updates the model using normalized group-relative advantages and a clipped, KL-regularized objective.

3 PRELIMINARIES

  • An ACWM generates a video from an initial frame and action chunk, and metric-based rewards compare that video with ground truth while IDM-based rewards compare inferred actions with conditioning actions.Metric examples include PSNR, SSIM, and MSE.Source: §3 · p. 3
  • Flow-GRPO samples groups of videos for each conditioning input, scores them with reward R, and normalizes scores into group-relative advantages.The normalized advantage uses the group reward mean and standard deviation.Source: §3 · p. 3
  • The ACWM update maximizes a clipped likelihood-ratio objective with a KL penalty toward a frozen reference model.The clipping range is ε, and β controls KL regularization.Source: §3 · p. 3

4 METHOD

The method constructs AVM-Eval to compare IDM action-following assessment on real and generated videos, then develops ACWM-RM as a reference-free reward for VQ and AF. Its training combines complementary mismatch preferences with real-video dynamics pretraining and preference-based scoring.

4.1.1 SOURCE DATA PREPARATION

  • The source data include AgiBotWorld subsets for IDM training, predictor pretraining, preference construction, and seen- and unseen-task evaluation, plus Genie-Sim, RoboDojo, and RoboTwin data for IDM generalization.AgiA covers 36 tasks; AgiB has 10.0k episodes across 30 tasks, and AgiC has 1.5k episodes across 10 tasks.Source: §4.1.1 · pp. 3–4

4.1 ACTION AND VIDEO MISMATCH BENCHMARK

  • AVM-Eval compares AF-assessment reliability on real and ACWM-generated videos using action-mismatch and video-mismatch pairs.The benchmark contains 67.2k pairs.Source: §4.1.2 · p. 4

4.1.2 AVM-EVAL CONSTRUCTION

  • Action-mismatch pairs hold a real video fixed and contrast its recorded action chunk with a perturbed one, probing numerical and semantic errors in actions.The benchmark has 53.9k such pairs and also perturbs data from other robot datasets to assess generalization across embodiments and scenes.Source: §4.1.2 · p. 4
  • Video-mismatch pairs hold actions fixed and rank generated videos using nDTW for AF, with filtering to limit visual-quality influence, because generated videos may not follow their conditioning actions.Pairs are formed within and across four ACWMs; the caption describes Figure 2 as covering both mismatch-pair constructions and reference-free reward computation.Source: §4.1.2 · p. 4, Figure 2 · p. 5

4.2 REWARD MODEL FOR ACTION-CONDITIONED WORLD MODELS

  • ACWM-RM is a reference-free reward model for jointly assessing VQ and AF, addressing the partial supervision of existing rewards.It scores a candidate video conditioned on an action chunk and associated state sequence.Source: §4.2 · p. 5

4.2.1 AVM-PREF CONSTRUCTION

  • AVM-Pref contains 506.3k complementary pairs: action-mismatch pairs provide AF supervision, while generated-video mismatch pairs provide VQ supervision and expose training to generated videos.The dataset uses numerical and semantic action perturbations and global and local PSNR and LPIPS for video-pair construction; combining both pair types improves AF assessment on generated videos.Source: §4.2.1 · p. 5

4.2.2 REWARD MODEL ARCHITECTURE AND TRAINING STRATEGY

  • ACWM-RM uses a frozen V-JEPA 2.1 encoder, an action-conditioned predictor, and a patch-weighted score head, with real-video dynamics pretraining followed by joint preference training.The score head predicts patch scores and spatial weight logits, applies softmax weights, and averages frame-level scores over future frames; paired candidates are ranked with a Bradley-Terry loss.Source: §4.2.2 · pp. 5–6

5 EXPERIMENTS

The experiments test whether IDM performance transfers from real to generated videos, then compare reward signals and ACWM-RM design choices. ACWM-RM leads the reported reward comparisons in visual quality and action-following fidelity, while ablations examine the roles of preference types and scoring design.

5.1 EXPERIMENTAL SETUP

  • The evaluation compares four architecturally distinct IDMs on AVM-Eval and reward alternatives in Flow-GRPO fine-tuning of DreamDojo-2B.The IDMs are trained on AgiTrain A; Flow-GRPO fine-tunes on AgiB and evaluates on AgiC.Source: §5.1 · p. 6

5.2 IDM PERFORMANCE ON AVM-EVAL

  • Among four IDMs, EVAI reaches 86.1% accuracy on real-video action-mismatch pairs but 70.0% on generated-video pairs; overall action-mismatch accuracy across embodiments and scenes ranges from 57.4 to 61.6.Strong action-mismatch performance on real videos does not ensure reliable evaluation of generated videos.Source: §5.2 · p. 7, Table 2 · p. 7, Table 3 · p. 7

5.3 FLOW-GRPO WITH DIFFERENT REWARD SIGNALS

  • ACWM-RM achieves the best results across all five VQ metrics and alone raises nDTW over Origin from 14.84 to 16.41.The VQ comparison is against the full-reference metric-based reward; the nDTW comparison is against Origin.Source: §5.3 · p. 7, Table 4 · p. 7, Table 5 · p. 8
  • ACWM-RM maintains physical-consistency scores comparable to Origin across all three criteria, while the VLM-based reward scores 3.86 versus Origin’s 4.00 overall.ACWM-RM does not explicitly reward physical consistency.Source: §5.3 · p. 8, Table 5 · p. 8

5.4 ABLATION STUDY

  • With the predictor frozen, the complete score head raises video-mismatch accuracy from 55.0 to 79.0, with most of the gain from learned comparison-based scoring and a further gain from patch weighting.Learned comparison-based scoring raises accuracy from 59.4 to 77.7; patch weighting brings it to 79.0.Source: §5.4 · p. 8, Table 6 · p. 9
  • Combining action- and video-mismatch preferences yields 91.6 accuracy on real-video pairs and 88.6 on generated-video pairs, versus 71.8 generated-video accuracy with action-mismatch preferences alone.The ablation attributes AF supervision to action-mismatch pairs and transfer to generated videos to video-mismatch pairs.Source: §5.4 · p. 8, Table 6 · p. 9
  • ACWM-RM exceeds JEPA-IDM by 3.1 points on action-mismatch pairs and 14.9 points on video-mismatch pairs, despite matching its backbone, training data, and two-stage training paradigm.The comparison uses feature prediction and scoring in ACWM-RM versus action prediction in JEPA-IDM.Source: §5.4 · p. 9, Table 6 · p. 9

6 CONCLUSION, LIMITATIONS, AND FUTURE WORK

AVM-Eval shows that strong action-mismatch discrimination on real videos does not ensure reliable AF assessment on generated videos; ACWM-RM performs better than alternative reward signals on VQ and AF in DreamDojo-2B fine-tuning. The paper also identifies limits in physical-consistency supervision and the scope of downstream evaluation.

6 CONCLUSION, LIMITATIONS, AND FUTURE WORK

  • ACWM-RM outperforms IDMs on generated-video AF and yields better VQ and AF than alternative reward signals when fine-tuning DreamDojo-2B.AVM-Eval also shows that strong action-mismatch discrimination on real videos does not ensure reliable AF assessment on generated videos.Source: §6 · p. 9
  • ACWM-RM does not explicitly supervise physical consistency, and downstream RL improvements have been demonstrated only for DreamDojo-2B on AgiBotWorld.Fine-tuning maintains physical-consistency scores close to those of the pre-RL model, but the reward does not directly target violations such as interpenetration.Source: §6 · p. 9

AI USE STATEMENT

  • The authors used generative AI to polish prose and check grammar, but not to generate research ideas; they reviewed the AI-assisted text and code.The authors also report responsibility for the final content, including AI-assisted text, code, and experimental results.Source: AI USE STATEMENT · p. 10

REPRODUCIBILITY STATEMENT

  • Training configurations and hyperparameters are provided in appendices for ACWM-RM, baseline IDMs, JEPA-IDM, and Flow-GRPO.Additional appendices cover action perturbations, preference-pair construction, evaluation prompts, and dataset statistics.Source: REPRODUCIBILITY STATEMENT · p. 10

A ACTION PERTURBATIONS FOR ACTION-MISMATCH PAIRS

  • Each clip has 12 consecutive action transitions, with each action comprising translation, rotation, and gripper channels for both arms.The clean action chunk is represented as a 12-by-14 array; the text also describes channel-wise numerical perturbations.Source: A ACTION PERTURBATIONS FOR… · p. 12

A.1 ACTIVE CHANNEL IDENTIFICATION

This section describes how mismatch pairs are constructed and filtered, using active action channels to guide perturbations and semantic edits and nDTW score gaps to identify video preferences. It also covers visual-quality controls that retain nDTW preferences only when video quality is comparable, assessed with four global and motion-local metrics.

A.1 ACTIVE CHANNEL IDENTIFICATION

  • Active channels are identified from 13-frame trajectory spread, with per-arm thresholds set at histogram valleys to exclude static and servo-noise channels.The six thresholds trade some false negatives for fewer false positives and remain fixed during pair generation.Source: §A.1 · pp. 12–13

A.2 NUMERICAL ACTION PERTURBATIONS

  • Action negatives are formed by applying selected perturbations to sampled windows of consecutive actions, and a pair is kept only when the negative differs from the positive and dist(a+, a−) ≥ 0.12.With probability 0.3, the whole chunk is one window; otherwise, one to three windows of one to seven consecutive actions are sampled, with neighboring windows separated by at least one unedited action. The perturbation pool includes channel-specific edits, Gaussian noise, and temporal-order permutation.Source: §A.2 · pp. 14, 16

A.3 SEMANTIC ACTION PERTURBATIONS

  • Semantic negatives edit episode state trajectories around detected grasp, release, or mid-motion events, producing position-offset, gripper-timing, and drift-or-freeze errors.Event detection uses gripper thresholds spanning the middle 40% of observed range; pairs require an edited action and normalized distance of at least 0.12.Source: §A.3 · pp. 16–18

B.1 NDTW-BASED PAIR FILTERING

  • Candidate video pairs are retained when their nDTW score difference reaches the 5th-percentile threshold: 0.03 for intra-model pairs and 0.12 for cross-model pairs.The thresholds were estimated from rollouts conditioned on 240 clips, with the higher-scoring video preferred.Source: §B.1 · pp. 18–19, Figure 5 · p. 18

B.2 VISUAL QUALITY CONTROL

  • Pairs retained by nDTW filtering keep their preference labels only when all four PSNR and LPIPS quality gaps fall within calibrated thresholds.For each metric and intra- or cross-model pair type, the threshold is the 2nd percentile of positive quality gaps from candidate pairs constructed from AgiBotWorld. Signed gaps are positive-minus-negative for PSNR and negative-minus-positive for LPIPS.Source: §B.2 · p. 19

C VIDEO-MISMATCH PAIR CONSTRUCTION FOR AVM-PREF

  • Video-mismatch pairs use global PSNR and LPIPS over entire future frames and local PSNR and LPIPS over motion-containing regions between transitions.Each clip shares its initial conditioning frame between ground-truth and generated videos. Metric values are averaged across future frames for global metrics and valid transitions for local metrics; higher PSNR and lower LPIPS indicate better quality.Source: C VIDEO-MISMATCH PAIR CONSTRUCTION FOR… · p. 19

C.1 GLOBAL VISUAL QUALITY METRICS

The appendix defines global and motion-focused local visual-quality metrics, then describes their use in selecting video-mismatch pairs. It also specifies validation handling and the action-conditioned predictor’s dynamics-pretraining setup.

C.1 GLOBAL VISUAL QUALITY METRICS

  • Global PSNR and LPIPS compare generated frames with corresponding ground-truth frames, averaging across all future frames.The global LPIPS implementation uses a fixed AlexNet checkpoint.Source: §C.1 · pp. 19–20

C.2 MOTION REGION EXTRACTION

  • Motion regions are extracted from optical flow in both videos, capturing missing ground-truth motion and spurious generated motion.Connected components become bounding boxes, with boxes smaller than 5 × 10^-4 of frame area discarded.Source: §C.2 · p. 20

C.3 LOCAL VISUAL QUALITY METRICS

  • Local scores aggregate box-level evaluations across valid transitions, omitting empty regions while retaining generated-video motion when ground truth is stationary.Local PSNR scores enclosed pixels; local LPIPS compares cropped regions.Source: §C.3 · p. 21

C.4 PAIR SELECTION

  • A video pair is retained only when the preferred video scores higher on both global and local PSNR and lower on both LPIPS metrics.Pairs failing any of the four inequalities are discarded.Source: §C.4 · p. 21

D REWARD MODEL IMPLEMENTATION DETAILS

  • Internal validation subsets from Agitrain are reserved for checkpoint selection and excluded from optimization, separately from Agitest evaluation.Agitest is used for AVM-Eval and does not participate in checkpoint selection.Source: D REWARD MODEL IMPLEMENTATION DETAILS · p. 21

D.1 DYNAMICS PRETRAINING

  • The 24-layer, 0.31B-parameter predictor uses block-causal attention and is pretrained with teacher-forcing and rollout losses before its weights initialize preference training.The rollout predictions are compared with encoder targets, and gradients pass through the first rollout prediction.Source: §D.1 · p. 23

D.2 PREFERENCE TRAINING

Preference training jointly updates a dynamics-pretrained predictor and a score head using paired preferred and dispreferred candidates. Candidate conditioning states are constructed from each candidate’s actions and initial state, and reward scoring does not require ground-truth future video.

D.2 PREFERENCE TRAINING

  • Preference training initializes the predictor from dynamics pretraining and trains it jointly with a randomly initialized score head using a pairwise Bradley-Terry objective.The loss applies the sigmoid to the preferred-minus-dispreferred score difference.Source: §D.2 · pp. 23–24
  • Action-mismatch candidates share a real video and initial state but use their own original or perturbed actions to construct consistent conditioning states.Video-mismatch candidates use different generated videos but share the original actions, initial state, and integrated states.Source: §D.2 · pp. 23–24
  • During Flow-GRPO, candidates under the same condition share actions and conditioning states, while ACWM-RM scores each candidate’s visual history and evaluated frames without future ground-truth video.The state sequence is integrated from the initial state and action chunk.Source: §D.2 · p. 24
  • The score head compares projected patch features using learned patch scores and weights, and its unbounded scalar output is used directly as the reward.The encoder remains frozen, while the pretrained predictor and score head are updated.Source: §D.2 · p. 24

D.3 TRAINING BUDGET ACCOUNTING

The budget accounting reports nominal clip-draw counts for dynamics pretraining and preference training. It distinguishes matching nominal budgets from matching compute or unique data exposure.

D.3 TRAINING BUDGET ACCOUNTING

  • Dynamics pretraining uses 4.08M clip draws, while three preference-training epochs yield about 3.04M candidate-clip draws.Each preference pair contributes two candidate-clip draws.Source: §D.3 · p. 24
  • The accounting counts repeated samples: one pretraining clip is one draw, and one preference pair contributes two candidate-clip draws.IDM draws likewise count complete clips covering all 12 action transitions.Source: §D.3 · p. 24
  • The nominal clip-draw budget approximately matches that of each IDM, but this does not establish equal computational cost or equal unique-video or frame exposure.Actual draw and step counts depend on held-out subsets and final partial batches.Source: §D.3 · p. 25

E TRAINING SETUP FOR THE BASELINE IDMS

The baseline IDMs are trained and selected on Agitrain, with a shared nominal clip-draw budget. Their reward compares predicted and conditioning actions after per-dimension standardization and Smooth-L1 loss.

E TRAINING SETUP FOR THE BASELINE IDMS

  • Four baseline IDMs are trained on Agitrain using their official implementations, with adaptations for bimanual actions.Their backbones and action interfaces differ, including a 14-dimensional output layout for TAP-IDM and a Jacobian-based action recovery for J-IDM.Source: E TRAINING SETUP FOR THE BASELINE IDMS · p. 25
  • Each IDM receives approximately 7.12M training clip draws on 13-frame clips with 12 action transitions, and its checkpoint is selected by validation action distance.The validation subset is internal to Agitrain.Source: E TRAINING SETUP FOR THE BASELINE IDMS · p. 25, Table 9 · p. 26
  • Predicted and conditioning actions are standardized with fixed per-dimension Agitrain statistics before reward computation.The same statistics are applied to both actions, so their means cancel in the residual.Source: E TRAINING SETUP FOR THE BASELINE IDMS · p. 25
  • The reward averages threshold-1 Smooth-L1 loss equally over 12 transitions and 14 action dimensions, producing a nonpositive scalar closer to zero for better action agreement.Standardization puts translation, rotation, and gripper errors on a dimensionless scale relative to Agitrain variability; Smooth-L1 is quadratic for small residuals and linear for large ones.Source: E TRAINING SETUP FOR THE BASELINE IDMS · p. 26

F PROMPT AND CRITERIA FOR EVALUATION

The evaluation prompt scores visible object, manipulator, and anomaly evidence in generated clips, while Flow-GRPO fine-tuning uses specified optimization settings. Evaluation on AgiC averages results over five generation seeds.

F PROMPT AND CRITERIA FOR EVALUATION

  • The evaluation prompt is used for VLM-based rewards in Flow-GRPO and for assessing artifacts and physical inconsistencies in generated videos.The prompt is from RBench; Table 11 identifies its uses with Qwen3-VL and GPT-5.6 sol.Source: F PROMPT AND CRITERIA FOR EVALUATION · p. 26, Table 11 · p. 27
  • The prompt rates object consistency, manipulator consistency, and anomalies on 1–5 scales using visible evidence across a 13-frame clip, without judging task completion.It checks object identity and structure, robot geometry and articulation, and events such as teleportation, duplication, or false grasps.Source: F PROMPT AND CRITERIA FOR EVALUATION · pp. 26–27, Table 11 · p. 27

G FLOW-GRPO OPTIMIZATION DETAILS

  • Flow-GRPO fine-tuning uses LoRA rank 32, AdamW at 3 × 10−4, 120 outer iterations, and 20 denoising steps per rollout.The diffusion noise level is 0.7, and ACWM-RM remains frozen during policy optimization.Source: G FLOW-GRPO OPTIMIZATION DETAILS · p. 27
  • Each sampled AgiB episode supplies a shared initial-frame and action condition for eight rollouts, with four rollout batches collected per outer iteration.The setup uses 64 rollouts per batch and four inner PPO-style epochs on collected data.Source: G FLOW-GRPO OPTIMIZATION DETAILS · p. 27
  • The policy optimization uses four PPO-style epochs, a 10−3 ratio clipping range, KL coefficient β = 0.004, and gradient clipping at 1.0.Gradients are accumulated over 16 rollouts.Source: G FLOW-GRPO OPTIMIZATION DETAILS · p. 27
  • Optimized ACWMs are evaluated on AgiC using five video-generation seeds, reporting means and standard deviations for visual quality, action-following fidelity, and physical consistency.Source: G FLOW-GRPO OPTIMIZATION DETAILS · p. 27

H DATASET AND PREFERENCE PAIR STATISTICS

This section describes dataset composition and preference-pair statistics, including how candidate availability constrains pair balancing. It also defines distance summaries for action- and video-mismatch pairs.

H DATASET AND PREFERENCE PAIR STATISTICS

  • AVM-Eval and AVM-Pref pair counts are organized by action perturbation type and video-mismatch model, model-pair, and task distributions.The dataset statistics draw on Agitrain, Agitest, AgiB, RoboT, Gen, RoboD, and AgiC; the cited task-statistics tables specify their episode and task counts.Source: H DATASET AND PREFERENCE PAIR STATISTICS · p. 28, Table 18 · p. 30, Table 19 · p. 30
  • Video-mismatch pair counts aim for balance across models and tasks, but eligible candidates limit task-level quotas and cross-model pairing requires rollouts under the same input condition.Quotas are redistributed to tasks with sufficient candidates.Source: H DATASET AND PREFERENCE PAIR STATISTICS · p. 31
  • Action-mismatch separation is summarized by standardized action differences across the full action chunk, while filtering distance averages only modified entries.The standard deviations are estimated from the corresponding source dataset; Figures 11 and 13 show distributions for AVM-Pref and AVM-Eval perturbations.Source: H DATASET AND PREFERENCE PAIR STATISTICS · p. 32, Figure 11 · p. 32
  • Video-mismatch separation in AVM-Eval is the nDTW score gap between positive and negative generated videos relative to their shared ground-truth video.Figure 12 reports this gap for intra-model and cross-model pairs on Agitest.Source: H DATASET AND PREFERENCE PAIR STATISTICS · p. 33, Figure 12 · p. 33
  • AVM-Pref video-mismatch separation combines four standardized quality-metric differences using their root mean square.The four metrics are PSNRglob, PSNRloc, LPIPSglob, and LPIPSloc; the same standard deviations are used for intra-model and cross-model pairs.Source: H DATASET AND PREFERENCE PAIR STATISTICS · p. 33
  • The reported video-mismatch distance distributions cover intra-model and cross-model AVM-Pref pairs on Agitrain.Figure 14 shows these distributions.Source: H DATASET AND PREFERENCE PAIR STATISTICS · p. 33

I JEPA-IDM IMPLEMENTATION DETAILS

JEPA-IDM adapts ACWM-RM’s predictor to action prediction and uses its inferred actions with an IDM-style distance score. It is pretrained on real video-action pairs before preference fine-tuning on AVM-Pref.

I JEPA-IDM IMPLEMENTATION DETAILS

  • JEPA-IDM shares ACWM-RM’s preprocessing, frozen visual encoder, and predictor backbone, but scores candidates with an IDM action-distance formulation instead of a learned scalar score head.Source: I JEPA-IDM IMPLEMENTATION DETAILS · pp. 33–34
  • Figure 13 presents action-mismatch distances across five source datasets and numerical versus semantic perturbations on logarithmic axes.Source: Figure 13 · p. 34
  • The predictor independently encodes clip frames and appends left- and right-arm readout tokens to each subsequent frame block under block-causal attention.During preference training and evaluation, visual blocks come from each candidate video, while its action chunk is used only to compute the score.Source: I JEPA-IDM IMPLEMENTATION DETAILS · pp. 34–35
  • A shared MLP maps each arm’s readout feature to seven action components, yielding 12 bimanual actions for a 13-frame clip.The components are translation, rotation-vector, and gripper-opening increments.Source: I JEPA-IDM IMPLEMENTATION DETAILS · p. 35
  • The predictor is pretrained on real Agitrain video-action pairs with standardized Smooth-L1 action loss while the visual encoder remains frozen.The same fixed action-component standard deviations are used in preference training.Source: I JEPA-IDM IMPLEMENTATION DETAILS · p. 35
  • Preference training initializes from action-prediction pretraining and fine-tunes the predictor on AVM-Pref with a Bradley–Terry loss.Trainable JEPA-IDM parameters use a peak learning rate of 2 × 10−5.Source: I JEPA-IDM IMPLEMENTATION DETAILS · p. 35
Loading 2610.32222v1…