IsoVRBench: Video Reward Modeling Without Human Feedback

1 Introduction

• Controlled interventions change one property while holding the rest fixed, so the preferred side and tested property follow from construction without human labels.Human validation is needed only once per intervention to confirm perceptibility; the same interventions can be regenerated as generators evolve.Source: §1 · p. 1
• IsoVRBench contains 8,000 pairs from eight interventions targeting motion, text alignment, visual quality, and overall quality.The pairs support reward-model evaluation, training, and video-generator post-training.Source: §1 · p. 2, Figure 1 · p. 2
• On shuffled videos, the best reward model reaches only 60.8% against 100% for humans, and existing benchmark rankings do not predict sensitivity to frame order.Across models, broad differences between generators are easier to detect than specific changes such as temporal order.Source: §1 · p. 2
• IsoVRM-Q2 beats same-base VideoReward by 3.8 points on VBench-2.0 despite VideoReward’s larger human-annotated training set.Intervention supervision also yields a generator preferred by human raters over the one post-trained with VideoReward.Source: §1 · p. 2
• Intervention-derived supervision can train competitive reward models without human preference labels or augment human data, improving downstream video-generator post-training.The paper reports gains in reward-model transfer and human preferences for post-trained generators.Source: §1 · p. 2

2.1 Controlled intervention framework

• Controlled interventions specify a preferred anchor and a counterpart changed along targeted dimensions; perceptibility, confinement, and plausibility are conditions for controlled evaluation.Confinement means changing only the target dimension, allowing reward differences to be attributed to that property.Source: §2.1 · pp. 2–3

2.2 Intervention definitions

• Frame-reuse edits reorder or repeat anchor frames, while re-rendering changes motion, prompt attributes, or denoising steps; generator substitutions target overall quality but give up confinement.Temporal Shuffle uses high-motion anchors, and Frame Hold repeats 8 to 12 frames; substitutions replace the anchor generator with weaker models.Source: §2.2 · p. 3

2.3 Human validation of perceptibility

• Human agreement is 98–100% on six interventions, 93% on Frame Hold, and 84% on Reduced Steps; raters identify all 64 control pairs as ties.Raters judged intervention and same-clip control pairs separately on visual quality, motion, and text alignment.Source: §2.3 · pp. 4–5, Table 2 · p. 5

2.4 Corpus and splits

• The benchmark contains 8,000 pairs, and the 164,000-pair training corpus uses disjoint prompts across training, validation, and benchmark splits.The training corpus has 160,000 training pairs and 4,000 validation pairs; prompt-level separation addresses video leakage from reused anchor clips.Source: Table 1 · p. 3, §2.4 · p. 5

• Humans average 96.4% across IsoVRBench, compared with 73.3% for the best reward model, VideoReward.Five of nine reward models are best on at least one intervention.Source: §3 · p. 5
• Rankings on published benchmarks do not predict Temporal Shuffle accuracy: none of the plotted correlations differs significantly from zero.The two general-purpose judges also show the frame-order weakness, scoring 58.9% and 57.1% on Temporal Shuffle.Source: Figure 3 · p. 6, §3 · p. 6
• 60.8% is VideoReward’s Temporal Shuffle accuracy, versus 100% for humans; six of nine reward models score at or below chance on that intervention.On Frame Hold, VideoReward reaches 71.6%, versus 93% for humans.Source: §3 · p. 6
• Models averaging 68.1%–70.1% differ by up to 15.2 points on individual interventions, so aggregate accuracy masks property-specific variation.The evaluation covers nine reward models and two general-purpose vision-language judges.Source: §3 · pp. 5–6

4.1 No published benchmark predicts frame order sensitivity

• Across published benchmarks, rank correlations with Temporal Shuffle range from −0.15 to +0.37 and none is significant; benchmark rankings do not indicate frame-order sensitivity.VideoReward performs best among reward models on Temporal Shuffle and Frame Hold despite middling or low rankings on the three video reward benchmarks.Source: §4.1 · pp. 6–7
• Existing benchmarks track generator identity or prompt alignment rather than frame order: GenAI-Bench ranks models like Weak Gen (5B), while TempCompass tracks Attribute Edit.The two generator identities alone predict 74.1% of GenAI-Bench labels.Source: §4.1 · p. 7

4.2 More frames do not improve temporal judgments

• Across eleven models, increasing frame count raises accuracy by 4.4 points on Frame Hold and 1.5 points on Temporal Shuffle; several models remain at or below chance on at least one intervention.VisionReward’s Temporal Shuffle accuracy falls as it receives more frames.Source: §4.2 · p. 7, Figure 4 · p. 7
• Intervention-pair training raises held-out Temporal Shuffle accuracy to 84.9% and 89.5%.The two values are reported for the trained models on held-out prompts.Source: §4.2 · p. 8

5.1 Training reward models

• 160,000 intervention pairs across eight interventions train the reward models, with 500 prompt-disjoint validation pairs per intervention and IsoVRBench prompts excluded.Both models use a Qwen2-VL-2B backbone and VideoReward architecture; IsoVRM-Q2 starts from initialization, while IsoVRM-VR starts from human-trained VideoReward weights.Source: §5.1 · p. 8
• On VBench-2.0, IsoVRM-Q2 scores 60.1 against VideoReward’s 56.3, but trails it on VideoGen-RewardBench, GenAI-Bench, and MJ-Bench-Video.IsoVRM-VR scores 63.5 against 56.3 on VBench-2.0, 73.3 against 73.2 on VideoGen-RewardBench, and 69.0 against 66.4 on MJ-Bench-Video; it trails on GenAI-Bench, 68.2 against 69.9.Source: §5.1 · p. 8

5.2 Training a video generator with intervention-supervised reward models

• The video generator is fine-tuned with GRPO for diffusion models using each reward model.The method is cited as Liu et al., 2025a.Source: §5.2 · p. 8
• In the three-way human study, the IsoVRM-Q2 generator wins 266 of 480 appearances (55.4%), versus 208 (43.3%) for the VideoReward-supervised generator.The IsoVRM-VR generator wins 246 appearances (51.2%).Source: §5.2 · p. 8
• On VBench-2.0, generators post-trained with IsoVRM-VR and IsoVRM-Q2 score 51.5 and 49.8, respectively, versus 49.0 with VideoReward.The two intervention-trained reward models therefore yield generators that improve on the VideoReward-supervised generator under both human judgments and VBench-2.0.Source: §5.2 · p. 9

• Reward-model evaluations commonly measure agreement with human choices on generated-video benchmarks (Liu et al., 2025b; Li et al., 2024; Tong et al., 2025).Source: §6 · p. 9
• Prior synthetic preference approaches include image distortions with human labels and generator comparisons that provide unlabeled preferences only for overall quality (Geng et al., 2025).Source: §6 · p. 9
• Temporal video-language benchmarks test video-text relations; TempCompass also uses conflicting clips differing in a temporal property (Cai et al., 2025).Source: §6 · p. 9

7 Conclusion

• Interventions reveal that current reward models see frame content but miss frame order, a failure none of five existing benchmarks reveals.The paper concludes that intervention pairs can train reward models and generators preferred by people, with or without human preference data.Source: §7 · p. 9

Ethics Statement

• Generated content and learned reward models may inherit biases, unsafe depictions, or representational gaps, so IsoVRBench is not a standalone measure of safety, factuality, or human values.Source: Ethics Statement · p. 9

A Accuracy over decided pairs

• Only VideoReward, VideoScore2, and LiFT keep their rank when accuracy excludes ties rather than counting them as wrong.VisionReward moves from seventh to second, while PickScore moves from second to fifth.Source: A Accuracy over decided pairs · p. 12, Table 5 · p. 12

B.1 Confinement has to be judged frame by frame

• Human raters preferred the Motion Re-gen anchor on 17 of 20 decided video pairs but on none of the 17 corresponding frame pairs.For frame-level checks, 41 of 45 regenerated-frame pairs were judged equal; all eight identical controls were also judged equal.Source: §B.1 · p. 12

B.2 Intervention strength sets difficulty

• At 8 rather than 16 denoising steps, eight models gain 7.4–14.2 points, while LiFT falls from 53.8 to 48.0 and is the only model below chance.Accuracy spread widens from 53.8–69.2 at 16 steps to 48.0–83.4 at 8; the 16-step setting is released as an additional split.Source: §B.2 · p. 13, Figure 5 · p. 13

C Temporal Shuffle anchor selection

• Temporal Shuffle uses the 1,000 anchors with the most motion; the benchmark anchors range from 18.3 to 56.1 mean absolute pixel differences, versus about 12 for a typical clip.The training corpus instead includes anchors above 25, computed on every fourth frame, which admits 23% of anchors.Source: C Temporal Shuffle anchor selection · p. 13

D Tie handling

• Tie rates range from zero to 61%, tracking score granularity; LiFT has ten distinct scores and ties on 61% of pairs.The paper counts ties as wrong because a tie gives no ordering for a difference people perceive.Source: D Tie handling · p. 13

E Prompted judge protocol

• Using expected digits rather than decoded digits reduces both prompted judges’ tie rates from as high as 47%–54% and 17%–20% to below 1%.The expected-digit method changes average rank under the tie rule but not the judges’ frame-order results.Source: E Prompted judge protocol · pp. 13–14

F Frames sampled per model

• Frame count orders neither accuracy column for the two frame-reuse interventions, and each model’s architecture prevents separating frame count from model effects.The paper varies frame count within each model elsewhere; the table’s accuracies count ties as wrong.Source: Table 6 · p. 14, F Frames sampled per model · p. 14

G Reward model training

• Both released reward models use the same training configuration, train on 160,000 pairs, and validate on 4,000 pairs drawn equally from eight interventions.Training, validation, and benchmark prompt sets are disjoint.Source: G Reward model training · p. 14, Table 7 · p. 14

H The trained reward models on IsoVRBench

• Because training excludes benchmark prompts but uses the same eight interventions, this evaluation tests new prompts rather than new interventions.Source: H The trained reward models on… · p. 15
• The two released models average 23.0 and 23.4 points above VideoReward on IsoVRBench, with at least 99.0% accuracy on six interventions.Their remaining errors are concentrated on Temporal Shuffle and Attribute Edit.Source: Table 8 · p. 15, H The trained reward models on… · p. 15
• VideoReward initialization raises Temporal Shuffle accuracy from 84.9% to 89.5%, while Qwen2-VL initialization scores 86.8% versus 85.5% on Attribute Edit.The two models’ average accuracies differ by 0.4 points.Source: H The trained reward models on… · p. 15

I Generator identity in preference benchmarks

• Generator identity alone predicts 74.1% to 78.2% of benchmark labels, exceeding VideoReward on each benchmark despite never seeing the videos.The predictor uses generator win rates against one another in the other folds of a five-fold split.Source: I Generator identity in preference… · p. 15, Table 9 · p. 15

J Generator ranking in VideoReward and in our reward models

• VideoReward rates both generators present in its training data above their human win rates and both absent generators below them on VBench-2.0.For CogVideo, its implied win rate is 40.9, versus 28.1 for human labels; with four generators, this pattern arises by chance one time in eight.Source: J Generator ranking in VideoReward and… · p. 16, Table 10 · p. 16
• Five same-generator interventions carry no generator ranking, while dropping generator substitutions costs IsoVRM-Q2 only 1.9, 1.6, and 0.5 points on three transfer benchmarks.The paper reports that the inherited generator ranking affects VideoReward on VBench-2.0, whose generators it partly never saw, but not on VideoGen-RewardBench, whose generators it saw in training.Source: J Generator ranking in VideoReward and… · p. 16

K Rank on overall quality does not predict rank on frame order

• PickScore ranks first on generator-substitution pairs but eighth on Temporal Shuffle at 45.0, while VideoReward ranks eighth on substitutions and first on Temporal Shuffle.Table 11 compares mean accuracy over three generator-substitution interventions with Temporal Shuffle accuracy; LiFT is the only model to hold its rank, at the bottom of both.Source: K Rank on overall quality does not… · p. 16, Table 11 · p. 16
• The two best models on generator-substitution pairs do not exceed chance on frame order.Without LiFT, correlations between overall mean accuracy and Temporal Shuffle and Frame Hold are +0.40 and +0.31, neither different from zero; with LiFT, they rise to +0.79 and +0.72, although LiFT is below chance on all eight interventions.Source: K Rank on overall quality does not… · pp. 16–17

L Frame counts the sweep cannot reach

• LiFT, VideoScore2, and VideoReward are omitted beyond 16, 16, and 32 frames, respectively, because of context, scoring-completion, and GPU-memory limits.LiFT's 4,096-position context pushes prompts out of context beyond 16 frames; VideoScore2 fails to score half the clips beyond 16 frames, and 48-frame VideoReward clips exceed 80 GB GPU memory at batch size one.Source: L Frame counts the sweep cannot reach · p. 17

M Caption ranking on temporal-understanding benchmarks

• VideoReward leads TemporalBench at 21.3, while IsoVRM-VR leads TempCompass at 69.7 and scores 18.4 on TemporalBench.The evaluation ranks complete captions for a fixed video using a model's text-alignment head or, for single-score models, its only output; exact ties count as wrong.Source: M Caption ranking on… · p. 17

P Human preference comparisons for the trained generators

• Both intervention-trained generators win more comparisons than the VideoReward-supervised generator; raters agree most on IsoVRM-Q2 versus VideoReward at 63.3% (κ=0.26).Agreement is lowest for IsoVRM-Q2 versus IsoVRM-VR at 55.8% (κ=0.11); choices are nearly transitive within each pass, with cycles on 5 of 240 prompt-pass pairs.Source: P Human preference comparisons for the… · p. 18, Table 16 · p. 20, Table 17 · p. 20

N Published benchmarks against IsoVRBench

• VBench-2.0 correlates negatively with Frame Hold at −0.61 and −0.80 without LiFT, while TemporalBench's largest reported correlation is +0.67 with Attribute Edit.Table 15 reports Spearman correlations over the nine reward models common to all columns; bolded correlations are separated from zero by an exact permutation test at p<0.05.Source: N Published benchmarks against… · p. 17, Table 15 · p. 19

O Every intervention against the human ceiling

• People score above the best reward model on every intervention, with differences exceeding the error of 45 rated pairs except Reduced Steps and Frame Hold.The two frame-reuse interventions are last by median reward-model accuracy; for them, median accuracy is 39.8 and 47.7, versus human agreement of 93 and 100.Source: O Every intervention against the human… · p. 18, Figure 6 · p. 19

Q VBench-2.0 by dimension for the trained generators

• Fourteen of eighteen VBench-2.0 dimensions separate the trained runs by less than 5 points, and removing Multi-View_Consistency alone reverses IsoVRM-VR's margin over VideoReward.The aggregate weights every dimension equally and gives equal weight to dimensions scored on 10 versus 264 videos; across dimensions, IsoVRM-VR leads on 11 and trails on 6, with median difference +0.8 and bootstrap mean-difference interval [−1.9, +7.8].Source: Q VBench-2.0 by dimension for the… · pp. 19–20, Table 18 · p. 20

R Generator training

• Each generator-training run uses 12,303 prompts, eight videos per prompt, and 30 sampling steps.Videos contain 121 frames at 704 × 1280; reward weights visual and motion quality by 0.5 and text alignment by 1.0, with a KL penalty to a reference policy updated every 10 steps.Source: R Generator training · p. 20