MJ-VIDEO: Fine-Grained Benchmarking and Rewarding Video Preferences in Video Generation

1. Introduction

• Existing video reward evaluations often focus on overall alignment, while fine-grained preference data remain limited and video generation still faces instruction, hallucination, safety, and bias challenges.Source: §1 · p. 1
• MJ-BENCH-VIDEO evaluates video preferences across five aspects using 28 fine-grained criteria.Source: §1 · p. 1
• MJ-VIDEO is a lightweight 2B Mixture-of-Experts reward model that decomposes assessment by aspect and routes text-video pairs to relevant reward objectives.Source: §1 · pp. 1–2
• MJ-VIDEO improves existing video reward models by 17.58% on overall and 15.87% on fine-grained video preference judgments, while preference tuning improves generated-video alignment.Source: §1 · p. 2

2. MJ-BENCH-VIDEO Benchmark

• MJ-BENCH-VIDEO introduces a multidimensional benchmark with five aspects and 28 fine-grained criteria for interpretable video-preference assessment.Source: §2 · p. 2

2.1. Overview of Evaluation Aspect Objectives

• Five dimensions assess instruction alignment, unsuitable content, visual fineness, spatial and action coherence, and bias in racial, gender, and age representation.Source: §2.1 · p. 3

2.2. Benchmark Curation

• Benchmark curation has three stages: data collection, filtering, and annotation.Source: §2.2 · p. 3, Figure 2 · p. 3

2.2.1. DATA COLLECTION

• Collection combines existing preference pairs, image-to-video conversion, and text-to-video generation, yielding 42,809 video pairs and 34,157 prompts.Source: §2.2.1 · p. 3

2.2.2. DATA FILTERING

• Filtering removes prompt-inconsistent, uniformly weak, tied-score, poor-quality, and overly similar pairs; the final benchmark contains 5,421 entries.Source: §2.2.2 · p. 3

2.2.3. DATA ANNOTATION

• Each pair receives 72 annotations across 28 criteria and five aspects, yielding 303,576 criterion scores, 54,210 aspect scores, and preference labels.Source: §2.2.3 · p. 4

3. MJ-VIDEO Reward Model

• MJ-VIDEO addresses the limits of frame-based vision reward models, which capture overall text-video alignment but not consistency, bias, or safety.It is designed to judge video preferences across diverse assessment criteria.Source: §3 · p. 4

3.1. Model Architecture

• Two stacked MoE layers route each text-video pair to five aspects, score 28 fine-grained criteria, and aggregate those scores into an overall preference score.Aspect routing weights combine the criteria scores; the resulting score is intended for general preference-tuning pipelines.Source: §3.1 · pp. 4–5

3.2. Multi-Stage Training

• Stage I trains the Criteria MoE to predict annotated fine-grained criteria scores.The labels comprise 28 criterion scores.Source: §3.2 · p. 5
• Stage II trains the Aspect MoE on aspect rankings from preference pairs while continuing criteria-score optimization; Stage III jointly trains both MoE layers using overall rankings.The third stage also incorporates the earlier losses and balances their impact with a hyperparameter.Source: §3.2 · p. 5

4. Experiment

• The study evaluates whether existing LVLMs and VideoLLMs judge preferences effectively, whether fine-grained training helps, whether MJ-VIDEO improves generation alignment, and what MoE adds.The dataset split is 4,336 training pairs and 1,085 test pairs; evaluated judges include open- and closed-source LVLMs and VideoScore, which lacks aspect-level evaluation.Source: §4 · p. 5, §4.1 · p. 5

4.1. Experimental Setup

• Experimental setup uses strict accuracy, which counts undecided cases as incorrect, and tie-aware accuracy, which gives identical scores partial credit.For video quality evaluation, the study uses accuracy and F1 because score distributions may be imbalanced.Source: §4.1 · p. 5

4.2. Fine-Grained Video Quality and Preference Evaluation Results

• Against similarly sized models, MJ-VIDEO improves accuracy by 20.12%, F1 score by 16.97%, and preference comparison by 51.67%; it also exceeds InternVL2-26B on those metrics.Against InternVL2-26B, the reported gains are 15.52% in accuracy, 9.05% in F1 score, and 45.86% in preference comparison; InternVL2-26B partially excels only in fineness evaluation.Source: §4.2 · p. 6

4.3. Overall Video Preference Evaluation Results

• MJ-VIDEO achieves the best test results on all three overall-preference datasets, improving over the best baseline by 17.58% on MJ-BENCH-VIDEO, 15.95% on Safesora-test, and 1.65% on GenAI-Bench.The comparison is summarized in Table 2.Source: §4.3 · p. 6, Table 2 · p. 7

4.4. MJ-VIDEO in Preference Alignment for Text-to-Video Generation

• With MJ-VIDEO as reward, preference-tuned VideoCrafter2 outperforms VideoScore and the original model in most evaluated aspects.Evaluation combines VBench measures of image quality, human action, scene composition, and overall consistency with human ratings of video quality and text-video alignment.Source: §4.4 · p. 7, Table 3 · p. 7

4.5. Ablation Study

• MJ-VIDEO improves over w/o Criteria MoE by 2.64% in average accuracy, 58.33% in F1, and 12.45% in strict preference accuracy, and over w/o Aspect MoE by 5.45% across three datasets.The study attributes the Criteria MoE gains particularly to Coherence and Consistency and Bias and Fairness, where the ablation shows strong biases.Source: Figure 4 · p. 7, §4.5 · p. 7

4.6. Case Study

• In one case, MJ-VIDEO recognizes an instructed ethereal bird that VideoScore overlooks; in another, it distinguishes a preferred video when both videos have similar quality.The authors attribute the second case to MJ-VIDEO’s training on pairwise preference data.Source: §4.6 · p. 7, Figure 5 · p. 8

Multimodal Judge

• Video reward research has been limited, particularly for fine-grained judgment, because temporal coherence complicates video evaluation.Prior work includes video-to-text feedback and fewer studies of text-to-video reward models and their evaluation.Source: §5 · p. 8
• CLIP-based judges offer reliable contrastive evaluations but often lack transparency, while LVLM-based judges offer flexible, more transparent feedback at greater computational cost.Both types are used in text-to-image and image-to-text tasks.Source: §5 · p. 8

Reward Model for Text-to-Video Generation

• Earlier video reward approaches include preference datasets without a practical reward model, scalar CLIP-based rewards with limited transparency, and a fine-grained MLLM model using pointwise QA and regression aggregation.The paper contrasts these approaches with MJ-BENCH-VIDEO and MJ-VIDEO, which provide fine-grained evaluation and aspect-specific judgments.Source: §5 · p. 8

6. Conclusion

• MJ-BENCH-VIDEO evaluates video generation across five aspects and 28 fine-grained criteria, while MJ-VIDEO uses specialized experts to decompose assessments.The paper concludes that MJ-VIDEO outperforms existing models and that the benchmark and reward model provide a framework for video-generation alignment.Source: §6 · p. 8

A. Annotation UI

• Human annotators use the interface to compare video pairs, edit generation prompts, and adjust criterion labels.Table 4 categorizes data sources as existing pairwise video preference, image-to-video conversion, and text-to-video generation.Source: A. Annotation UI · p. 12, Table 4 · p. 12, Figure 6 · p. 12

B. Prompt Design for Video Quality Assessment

• The framework standardizes video comparisons with a structured prompt and predefined criteria for consistent quantitative assessment.Assessors judge video quality based on category and subcategory descriptions.Source: B. Prompt Design for Video Quality… · p. 12, §B.1 · p. 13

B.1. General Evaluation Prompt

• The general prompt asks assessors to rate whether a generated video would be preferred by humans on a ten-level scale from “Extremely Poor” to “Outstanding.”Source: §B.1 · p. 13

B.2. Descriptions for Categories and Subcategories

• Alignment evaluates how accurately video content represents the objects, attributes, actions, counts, and spatial locations specified in captions.Source: §B.2 · p. 13, Table 5 · p. 13
• Safety assessment flags potentially inappropriate content, including crime, shocking or disgusting scenes, and NSFW elements.Source: §B.2 · p. 13, Table 6 · p. 14
• Fineness assesses visual quality by checking for distortions, blurring, and other artifacts.Source: §B.2 · p. 13, Table 7 · p. 14
• Coherence and Consistency checks spatial, temporal, and visual consistency, including continuity of actions and stable lighting or object placement.Source: §B.2 · p. 13
• Bias and Fairness examines possible representation biases related to gender, age, job, race, or education.Source: §B.2 · p. 14

• MJ-VIDEO achieves the best tie-aware performance across most aspects.Source: C. Tie-Aware Metric for Aspect-Level… · p. 14
• Bias & Fairness has relatively little test data, which may allow models assigning the same scores to videos to obtain higher tie-aware scores.Source: C. Tie-Aware Metric for Aspect-Level… · p. 14
• For Bias & Fairness, the authors consider the strict metric a more reliable indicator than the tie-aware score.Source: C. Tie-Aware Metric for Aspect-Level… · p. 14

D. Criterion-Level Evaluation

• Criterion-level annotations are used to identify strengths and weaknesses in model judgments within each aspect.Source: D. Criterion-Level Evaluation · p. 14
• Tables 11–15 report MJ-VIDEO and baseline results across individual criteria.Source: D. Criterion-Level Evaluation · p. 14, Table 8 · p. 15

E. Detailed Abliation Study on Aspect

• MJ-VIDEO outperforms the ablated model on accuracy, F1, and strict evaluation across most aspects.The authors attribute improved generalization and robustness to adversarial distributional biases to the MoE architecture.Source: E. Detailed Abliation Study on Aspect · p. 15

F. Experimental Details

• The section introduces experimental setup and training parameters, without specifying them in this passage.Source: F. Experimental Details · p. 15

F.1. Training MJ-VIDEO

• MJ-VIDEO is trained in three stages: criteria scoring, aspect routing, and joint training, with progressively broader annotations and trainable components.Criteria scoring trains the language model and regression layer; aspect routing uses aspect annotations; joint training uses overall preference annotations. Figure 7 compares MJ-VIDEO with the “w/o Criteria MoE” ablation.Source: §F.1 · pp. 15, 18, Figure 7 · p. 18

F.2. Preference Alignment for Text-to-Video Generation

• For preference tuning, VideoCrafter2 is fine-tuned with VADER on 5,000 VidProM prompts, using VideoScore and MJ-VIDEO separately as reward models.Training uses eight frames and runs for two epochs; VBench evaluation generates four videos per prompt, totaling 3,784 videos per text-to-video model.Source: §F.2 · p. 18

G.1. Case Study For Video Reward Modeling

• MJ-VIDEO prefers the better-aligned video when quality and alignment conflict, whereas VideoScore favors the higher-quality but less-aligned video.The paper attributes the difference to MJ-VIDEO’s gating layer, which dynamically weights dimensions using the video and prompt; VideoScore sums dimension scores.Source: Figure 8 · p. 19, §G.1 · p. 19
• MJ-VIDEO distinguishes a face-quality difference that InternVL2-26B misses, returning a preference for video 2 where InternVL2-26B returns a tie.The paper attributes MJ-VIDEO’s advantage in this case to training focused on visual details, particularly human features.Source: Figure 8 · p. 19, §G.1 · p. 19

G.2. Case Study For Text-to-Video Generation

• In the cat-and-piano example, the MJ-VIDEO-tuned cat appears more realistic and faces the piano, better matching the intended scene.The passage presents this as an example of improved scene alignment.Source: §G.2 · p. 19, Figure 9 · p. 20
• The MJ-VIDEO-tuned xylophone has detailed key structures, which the authors associate with higher visual fidelity and overall video quality.The case is described as illustrating gains in realism and detail fidelity.Source: §G.2 · p. 19, Figure 9 · p. 20
• In the single-dog case, MJ-VIDEO fine-tuning produces one dog as requested, while VideoScore produces two, missing the prompt specification.The comparison identifies prompt alignment as a benefit of MJ-VIDEO preference tuning.Source: §G.2 · p. 19, Figure 9 · p. 20
• Both saxophone videos have structural issues, but the MJ-VIDEO-tuned output more closely resembles real-world appearance and has greater clarity and overall quality.The comparison notes improvement despite structural issues in both outputs.Source: §G.2 · p. 19, Figure 9 · p. 20