RubricRM: Generative Reward Modeling via Dynamic Rubrics for Image Generation and Editing

1 Introduction

• RubricRM addresses the mismatch between fixed evaluation dimensions and instructions requiring distinct criteria, such as physical plausibility for realistic prompts and novelty for creative prompts.Existing generative reward models typically use predefined dimensions, while scalar-score models offer little interpretability.Source: §1 · p. 1
• For each instruction and candidate pair, RubricRM creates a rubric covering task intent, atomic dimensions, dimension weights, and scoring criteria, then aggregates weighted dimension scores into a preference.The rubric serves as an intermediate evaluation protocol between the instruction and preference decision, making the criteria inspectable.Source: §1 · p. 1
• RubricRM uses supervised fine-tuning on synthesized rubric trajectories to learn rubric-based scoring, followed by GRPO with dimension-level rewards to refine scoring accuracy.The rewards align predicted and reference score differences for each dimension and penalize reversed preference directions more strongly.Source: §1 · pp. 1–2, Figure 1 · p. 2
• To support rubric diversity, the authors classify instruction types, balance sampling, synthesize prompts for underrepresented categories, collect expert preferences, and create 31.8K generation and 30.7K editing training samples.Rubric trajectories are synthesized for each preference sample.Source: §1 · p. 2
• RubricRM-Gen-9B achieves accuracies of 72.00 on MMRB2, 74.12 on GenAI-Bench, and 84.45 on GenAI-Bench-Verified; RubricRM-Edit-9B reaches 75.40 on MMRB2, 46.40 on EditReward-ERB, and 85.64 on EditScore-ERB.The paper reports that these models outperform existing reward-model baselines across generation and editing tasks.Source: §1 · p. 2
• The authors develop specialized RubricRM models for both text-to-image generation and image editing, combining rubric-trajectory SFT with dimension-level GRPO.The models use Qwen3.5-4B and Qwen3.5-9B backbone models.Source: §1 · p. 2

Text-to-image generation evaluation

• Text-to-image evaluation has shifted from alignment metrics toward preference-trained rewards, but scalar outputs limit evidence and VisionReward uses predefined dimensions and weights.CLIPScore (Hessel et al., 2021) measures image-text similarity but is unreliable for compositional semantics (Jiang et al., 2024); ImageReward (Xu et al., 2023), HPS v2 (Wu et al., 2023b,a), and PickScore improve human alignment while retaining scalar rewards, and VisionReward uses predefined dimensions and weights.Source: §2 · pp. 2–3
• MLLM-based generative reward models add evaluative reasoning or context adaptation, yet existing criteria remain fixed or lack explicit requirements and weights.UnifiedReward (Wang et al., 2025) supports multiple multimodal reward tasks, UnifiedReward-Think (Wang et al., 2026b) adds explicit reasoning with fixed criteria, and UnifiedReward-Flex (Wang et al., 2026a) adapts assessment to context without specifying detailed criterion requirements or weights.Source: §2 · p. 3

Image editing evaluation

• Editing evaluation must balance instruction satisfaction with preservation of irrelevant content, while prior systems do not provide instruction-specific weighted, graded rubrics.EditReward (Wu et al., 2025) is a discriminative scalar model trained on over 200K editing preference pairs, and EditScore (Luo et al., 2025) uses high-fidelity generative rewards for online RL without instruction-specific rubrics; RubricRM targets edit execution, source preservation, and visual quality.Source: §2 · p. 3

Rubric-based evaluation

• Prior rubric methods use shared rules, external scoring, proxy transfer, or holistic decisions; RubricRM generates weighted, graded criteria and per-dimension scores for both candidates end to end.AutoRubric (Rao and Callison-Burch, 2026), RubricHub (Li et al., 2026), OpenRubrics (Liu et al., 2025), AutoRubric-R1V (Jia et al., 2025), Proxy-GRM (Qiu et al., 2026), AutoRubric-T2I (Kao et al., 2026), and Auto-Rubric as Reward (Tian et al., 2026) cover task-adapted rubric generation, rubric resources, multimodal rewards, shared rules with external frozen-VLM scoring, and prompt-conditioned binary criteria with holistic preference decisions. Unlike these approaches, RubricRM is a standalone, self-contained generative reward model that produces input-specific dimensions, normalized weights, graded criteria, and explicit scores in one inference pass.Source: §2 · p. 3

3.1 Task Definition

• For text-to-image and editing, RubricRM compares two candidates against the instruction, producing a rubric and dimension-level assessments whose weighted scores determine preference.The rubric specifies evaluation dimensions, their relative weights, and scoring criteria; editing inputs also include a source image.Source: §3.1 · pp. 3–4

3.2 Rubric Trajectory Data

• Multi-label classification and stratified sampling balance coverage across instruction categories, while identifying underrepresented categories for targeted prompt synthesis and SFT/RL splits.The taxonomy spans multiple dimensions, allowing prompts to carry multiple labels rather than forcing each sample into one category.Source: §3.2 · p. 4
• Text-to-image training uses 31,835 preference samples, including supplementary prompts rendered by ten models and retained only when three experts unanimously favor a pair by over 5 points.The editing set uses 30,695 samples from EditReward-Data, without additional prompt synthesis.Source: §3.2 · p. 4
• Gemini 3.1 Pro converts human preference labels into trajectories covering task intent, weighted dimensions, scoring descriptors, and candidate assessments; quality checks remove malformed or label-inconsistent trajectories.Conditioning synthesis on the human label anchors the rubric trajectory to that preference rather than using the teacher as an independent preference annotator.Source: §3.2 · p. 5

3.3.1 Stage 1: Supervised Fine-Tuning

• Supervised fine-tuning teaches the complete rubric-based judgment pipeline, from instruction analysis and dimension weighting to dimension-level scoring and a final preference.Text-to-image and editing models use Qwen3.5-4B and Qwen3.5-9B backbones, respectively.Source: §3.3.1 · p. 5, §3.3 · p. 5

3.3.2 Stage 2: GRPO with Dimension-Level Rewards

• GRPO follows SFT because trajectory imitation alone does not directly optimize preference accuracy; dimension-level rewards also distinguish correct directions from score-gap errors.The reward uses direction factors of 1.0 for matching signs, 0.6 when exactly one gap is zero, and 0.1 for opposite nonzero signs, alongside a score-gap magnitude factor.Source: §3.3.2 · pp. 5–6
• During GRPO, the synthesized rubric is fixed so candidate scores can be compared with reference scores under consistent dimensions; at inference, the model generates the rubric and scores candidates end to end.Groups with reward standard deviation below 0.05 are dynamically filtered to avoid amplifying noise in low-spread rollouts.Source: §3.3.2 · p. 6

4.1 Experimental Setup

• Evaluation covers MMRB2, GenAI-Bench and its expert-verified subset for generation, plus MMRB2, EditReward-ERB, and EditScore-ERB for editing.GenAI-Bench-Verified retains pairs where three expert annotators agree and match the original label. EditScore-ERB contains 3,072 preference pairs across 13 subtasks and reports Prompt Following, Consistency, Overall Quality, and Avg.Source: §4.1 · pp. 6–7

4.2 Experimental Results

• RubricRM-Gen-9B leads reward-model baselines on all three text-to-image benchmarks: 72.00 (MMRB2), 74.12 (GenAI-Bench), and 84.45 (GenAI-Bench-Verified).Compared with UnifiedReward-Flex-8B, it improves by 2.80, 0.72, and 0.25 points, respectively. RubricRM-Gen-4B scores 70.50, 73.20, and 83.11 despite its smaller backbone than several general MLLM baselines.Source: Table 2 · p. 7, §4.2 · pp. 7–8

4.1 Experimental Setup

• Baselines include general MLLM judges and specialized reward models, and separate generation and editing RubricRM models use Qwen3.5-4B and Qwen3.5-9B backbones.Training uses rubric-trajectory SFT followed by GRPO, with disjoint SFT and RL splits; the RL split contains approximately 15K samples per task.Source: §4.1 · p. 7

4.2 Experimental Results

• RubricRM-Edit-9B leads reward-model baselines on MMRB2 (75.40), EditReward-ERB Avg (46.40), and EditScore-ERB Avg (85.64), and leads EditReward-ERB at K=2 and K=4.RubricRM-Edit-4B leads at K=3 and achieves the highest EditScore-ERB Prompt Following (88.67) and Overall Quality (88.85); RubricRM-Edit-9B leads Consistency (79.66) and Avg. These results span instruction following, source-image consistency, and visual quality.Source: Table 3 · p. 8, §4.2 · p. 8

4.3 Ablation Study: Two-Stage Training

• Rubric-trajectory SFT provides the largest gains: 6.8–13.1 points for 4B and 3.4–10.7 for 9B on generation, and 6.5–11.1 and 7.6–13.1 on editing, respectively.Dimension-level GRPO adds smaller but consistent gains across benchmarks, calibrating rubric application by aligning predicted score gaps with the reference trajectory; SFT teaches criterion selection, weight assignment, and structured dimension-level scoring.Source: Table 4 · p. 8, Table 5 · p. 8, §4.3 · pp. 8–9

• RubricRM first determines what to evaluate through dynamic rubric generation, then applies criteria using weighted per-dimension scores.Source: §5 · p. 9
• Across generation and editing benchmarks, RubricRM improves over prior specialized reward models while preserving an inspectable judgment trace.Source: §5 · p. 9
• Its two-stage training uses SFT for rubric formulation and GRPO for scoring alignment.Source: §5 · p. 9

Limitations

• RubricRM may inherit proprietary-teacher biases and currently evaluates only static images, leaving video generation and temporal editing unaddressed.Future work will explore teacher-free rubric improvement, human-in-the-loop rubric correction, and extensions to video assessment.Source: Limitations · p. 9

Ethical Considerations

• Human preference labels and teacher-synthesized rubric trajectories may encode subjective or systematic biases that affect rubrics and preference judgments.Source: Ethical Considerations · p. 9
• Explicit dimensions, weights, and scores improve inspectability but are neither objective standards nor guaranteed causal explanations of decisions.Source: Ethical Considerations · p. 9
• Biased reward signals may be amplified when RubricRM optimizes downstream text-to-image generation or image-editing models.Source: Ethical Considerations · p. 9
• Audit RubricRM on the target content distribution and treat its rubric outputs as model judgments, not authoritative assessments.Source: Ethical Considerations · p. 9
• Because visual preference datasets may contain people or sensitive content, follow their access conditions and consider content-specific audits before downstream deployment.Source: Ethical Considerations · p. 9

A Method Details

• The appendix reports data-coverage and taxonomy statistics, the GRPO advantage-filtering rule, and representative dimension-level reward cases.Source: A Method Details · p. 11

A.1 Training Data and Taxonomy Statistics

• Stratified RL-split sampling kept maximum secondary-label coverage drift below 0.5 percentage points in both tasks while retaining low-frequency categories.The split was iteratively adjusted by swapping examples when secondary-label coverage deviated from its target.Source: §A.1 · p. 11
• Primary-category coverage spans both tasks, led by Common Objects (92.4%) for text-to-image and Object Editing (45.9%) for image editing.Long-tail text-to-image categories include World Knowledge (15.8%), Logical Reasoning (9.0%), and Text Rendering (6.5%); Figure 2 visualizes both task distributions.Source: §A.1 · p. 11, Figure 2 · p. 12
• Text-to-image co-occurrences often combine subject, style, and composition, motivating instruction-specific rubric dimensions.Common Objects+Style occurs in 63.6% of examples, while Common Objects+Composition & Perspective occurs in 49.5%.Source: §A.1 · p. 12
• Aggregated rubrics emphasize task-relevant criteria: subject fidelity, visual quality, composition, style, and prompt alignment for generation, plus source preservation and edit integration for editing.Dimension titles were grouped semantically, and the appendix reports their shares and average teacher-assigned weights in Tables 8 and 9.Source: §A.1 · p. 12

A.2 Fixed-Rubric Rollout during GRPO

• Fixed teacher rubrics make dimension-level rollout rewards comparable to reference score gaps and avoid noise from changing evaluation criteria.The rubric is conditioning context during GRPO, which scores dimensions rather than only the final binary preference.Source: §A.2 · p. 12
• Rubric generation is learned from complete SFT trajectories and performed end to end at inference, while GRPO calibrates rubric-conditioned scoring.No reference rubric is provided at inference; the fixed-rubric restriction applies only during GRPO.Source: §A.2 · pp. 12–13

A.3 GRPO Objective and Saturated-Group Filtering

• The GRPO appendix specifies dimension-level rewards, group-relative advantage normalization, saturated-group filtering, and a KL term against a reference policy.It reports training-stability curves for saturated-group filtering in Figure 3, without providing curve values in these passages.Source: §A.3 · p. 13

A.4 Dimension-Level Reward Case Studies

• Reversed dimension directions receive the strongest discount, tie mismatches a partial discount, and correct directions still earn reward for score-difference magnitude.Table 10 reports the per-dimension reward terms and final weighted reward for these cases.Source: §A.4 · p. 13, Table 10 · p. 14
• In the “writer creates stories, Overhead shot” case, both reference and rollout prefer B, but the rollout reverses the Thematic Depiction direction from A > B to A < B.A final-preference-only reward would not expose this incorrect scoring rationale.Source: §A.4 · p. 13, Table 10 · p. 14
• For “Person lying down with legs up, next to a dog,” Rollout 1 preserves the reference tie on Anatomical Correctness, while Rollout 2 introduces an unsupported A > B direction.Both rollouts select the correct final preference A.Source: §A.4 · pp. 13–14
• Table 11 reports percentage results on EditScore-ERB across Prompt Following, Consistency, Overall Quality, and overall, with best and second-best reward-model scores distinguished.The supplied caption does not provide cell values or identify a winning model.Source: Table 11 · p. 14
• Rollout 2 introduces an unsupported A > B direction, and this tie mismatch applies λtie = 0.6, lowering aggregate reward from 0.950 to 0.760.Source: §A.4 · p. 14
• In the speaker-and-snail case, the rollout selects the correct final preference A and matches every dimension’s direction, but predicts a Spatial Obstruction margin of 1/0 instead of 4/0.The case illustrates that a correct direction can still have an inaccurate score-difference magnitude.Source: §A.4 · p. 14, Table 10 · p. 14

Detailed editing results

• RubricRM remains competitive across subject, appearance, scene, and advanced editing categories rather than improving only one aspect.Table 11 expands the EditScore-ERB results by reporting Prompt Following, Consistency, and Overall Quality for each task category.Source: §B.1 · p. 14

Label-only supervision ablation

• Label-only SFT improves over base models but trails rubric-based SFT on most benchmarks, by 3.3–5.1 points (4B) and 1.9–3.9 points (9B) in text-to-image.It also falls behind on MMRB2 and EditReward-ERB for image editing; rubric trajectories teach models to decompose instructions into evaluation criteria before pairwise judgment.Source: §B.1 · pp. 14–15, Table 12 · p. 15

Saturated-group filtering analysis

• Filtering low-standard-deviation rollout groups stabilizes GRPO reward mean, response length, and entropy, whereas unfiltered training shows late collapse, growth, and drift.When rollouts receive nearly identical rewards, within-group normalization can amplify small numerical fluctuations into large policy-gradient updates; filtering suppresses this instability.Source: §B.1 · p. 15, Figure 3 · p. 15

B.2 Analysis of Generated Rubrics

• Rubrics for repeated instructions show dimension similarities of 0.763 for generation and 0.800 for editing, with weight differences of 0.102 and 0.110.The analysis samples five rubrics per instance using temperature 0.7 and top-p = 0.9, then matches dimensions by semantic similarity.Source: Table 13 · p. 16, §B.2 · p. 16
• For generation and editing, rubric similarity follows same instruction > same type > different type, with same-type similarities of 0.597 vs. 0.563 and 0.651 vs. 0.612, respectively.The selected evaluation dimensions also change according to task requirements.Source: Table 13 · p. 16, §B.2 · p. 16

B.3 Sensitivity to Teacher Choice

• Teacher choice affects rubric dimension selection and granularity, but not the underlying preference label inherited from human-annotated data.The teacher converts each fixed preference into a rubric trajectory rather than acting as an independent preference annotator.Source: §B.3 · p. 16
• Across teacher pairs, 91.5% of comparisons share a semantic dimension, 62.0% cover at least half of the smaller rubric, and average overlap is 0.534.The analysis holds inputs and human preference labels fixed across three teacher families; the resulting agreement leaves room for differences in criteria and granularity.Source: Table 14 · p. 16, §B.3 · p. 16

C Prompt Templates

• The inference prompt templates require task analysis, dynamic dimension selection, dimension-level scoring, weighted aggregation, and final preference selection.Source: C Prompt Templates · p. 16

D.1 Text-to-Image Generation

• For the cranberry-harvest prompt, the rubric weights prompt adherence and visual realism at 30% each, and composition and logical consistency at 20% each.The task analysis identifies the flooded bog, visible red berries, V-shaped geese, and autumn setting as key requirements.Source: §D.1 · p. 19
• On the weighted rubric, Image A scores 4.00 versus Image B’s 1.10 and is the closer realization of the cranberry-harvest prompt, with realistic details, autumn lighting, and logical composition.Image B is described as weaker in realism and logic.Source: §D.1 · p. 19

Image Editing

• The task requests a colorful kindergarten educational poster centered on the toy airplane, with a title, fun fact, safety message, and bold question placed in specified locations.The title goes at the top, the fact on the left, the safety message on the right, and the question at the bottom.Source: §D.2 · p. 20
• The rubric weights text accuracy and placement at 40%, thematic style and aesthetics at 30%, and preservation of the original airplane at 30%.Text evaluation checks that all four elements are present, correctly spelled, and in their requested locations; preservation checks that the airplane remains the centerpiece without unwanted artifacts.Source: §D.2 · p. 20
• 4.00 vs 3.10 weighted total scores favor Image A, which earns 4/4 on all dimensions while Image B scores 4/4 for text, 3/4 for aesthetics, and 2/4 for preservation.The evaluation prefers Image A.Source: §D.2 · p. 20