OmniVerifier-C1: Decomposed Graded Verifier for Rule-Governed Generative World Modeling

• Binary verifiers’ True/False feedback masks quality differences, weakens within-group reinforcement-learning signals, and may encourage shortcut learning and reward hacking.Source: §1 · p. 2
• OmniVerifier-C1 decomposes complex prompts or environmental rules into sub-questions, reasons over each component, and scores the proportion satisfied to capture partial correctness.Its structured outputs are reported to improve out-of-domain multimodal critic capabilities.Source: §1 · p. 2
• In rule-governed world modeling, pairwise verification against ground-truth states provides more accurate critiques than pointwise verification, making reference states consequential.Experiments across Maze and Sudoku support the verifier’s role in improving reinforcement-learning performance.Source: §1 · pp. 2–3
• Decoupled RL jointly optimizes sub-question binary judgments and global alignment scores, using precise 0/1 local feedback to support reliable global scoring.Source: §1 · pp. 2–3
• The verifier improves out-of-domain multimodal critic capability and reinforcement-learning performance in both text-to-image generation and rule-governed world modeling.Experiments also find that pairwise verification is more accurate than pointwise verification in rule-governed environments when ground-truth states are available.Source: §1 · pp. 2–3

2.1 Generative Verifier or Reward Models

• Existing generative verifiers offer interpretable rationales but retain coarse binary judgments; this work uses decomposed verification and Decoupled RL to produce continuous, optimization-friendly rewards.Generative reward models provide explicit rationales rather than scalar scores, while traditional reward models assign scalar scores.Source: §2.1 · p. 3
• Prior advances include ViVerBench and universal verification (OmniVerifier, Zhang et al., 2025b), meta-verification (DeepSeekMath-V2, Shao et al., 2025), and symbolic spatial rationales (OmniVerifier-M1, Zhang et al., 2026d).OmniVerifier-M1 uses annotations such as bounding boxes, points, or lines as verifiable rationales.Source: §2.1 · p. 3

2.2 Generative Model as World Reasoner

• World-modeling efforts include Emu3.5’s long-horizon next-state prediction and UnityVideo’s unified multimodal training; this work instead targets RL reward reliability in rule-governed worlds.Benchmarks such as WISE (Niu et al., 2025) and MMGR (Cai et al., 2025) assess cultural and scientific knowledge, and spatial, temporal, physical, and logical reasoning.Source: §2.2 · p. 3

3.1 Theoretical Advantage of Decomposed Graded Rewards for Generative RL

• The verifier decomposes complex prompts into sub-questions, judges each separately, and aggregates local decisions into a continuous score that preserves partial correctness.This yields denser, more optimization-friendly reinforcement-learning feedback than a single binary judgment.Source: §3.1 · p. 4

Experiment Setup.

• On ViVerBench, the Decomposed Graded Verifier outperforms the Discrete Binary Verifier in both RL-only and SFT+RL comparisons.The authors attribute the difference to the supervision signal rather than the training pipeline: decomposition and fine-grained judgments provide richer feedback and support generalization to out-of-distribution scenarios.Source: Table 1 · p. 5, Experiment Setup · p. 6

3.1 Theoretical Advantage of Decomposed Graded Rewards for Generative RL

• The continuous reward ranks samples perfectly by satisfied sub-questions, whereas the discrete reward ranks imperfect samples with different quality levels below perfectly.The result assumes the prompt can be losslessly decomposed and that at least two imperfect samples have different quality levels.Source: §3.1 · p. 5

3.2 Graded Verification Enhances Multimodal Reasoning and Critique Capabilities

• Binary outcome-only supervision can let multimodal verifiers exploit shallow visual patterns or shortcut cues, producing apparently correct verdicts without genuine reasoning and risking reward hacking (Wang et al., 2026c; Shao et al., 2025).Source: §3.2 · p. 5

Experiment Setup.

• Decoupled RL jointly optimizes global alignment scoring and binary judgments on individual sub-questions, using K + 1 samples per prompt-image pair.The score reward is 1 − |Ŝ − S*|, while each judgment receives reward 1[Âk = A*k]; precise local feedback supports fine-grained verification and global scoring.Source: Experiment Setup · pp. 5–6

4.1 Verifier-Guided RL for Traditional Text-to-Image Generation

• For traditional text-to-image generation, the verifier decomposes each prompt into fine-grained sub-questions and averages binary judgments as the reinforcement-learning reward.Each judgment assesses whether the generated image satisfies its corresponding sub-question.Source: §4.1 · p. 6

4.2 Verifier-Guided RL for Rule-Governed World Modeling

• Unlike text-to-image prompts with multiple plausible outputs, rule-governed tasks often determine a unique final state, enabling ground-truth images to support more accurate verifier rewards.The ground-truth image is introduced as an additional visual input to simplify verification and improve reward reliability.Source: §4.2 · pp. 6–7
• In world modeling, the generator predicts a resulting image from an initial visual state and rule-based instruction, with sub-questions assessing individual rules or goals.Pointwise verification judges the output against target states inferred from the initial state and rules.Source: §4.2 · p. 7
• Pairwise verification provides a substantially more reliable reward mechanism than pointwise verification by avoiding target-state inference and its associated reasoning errors.Pointwise verification infers a target state from the initial state and rules before judging the output; pairwise verification compares it directly with the ground-truth image, reducing score deviation to judgment error.Source: §4.2 · pp. 7–8

5.1 Experiment Setup

• The experiments use ShareGPT-4o-Image prompts for text-to-image tasks and six game-based tasks spanning planning, logic, construction, pattern, character, and spatial reasoning.Stable-Diffusion-3.5-medium and FLUX.1-dev serve as text-to-image backbones, Qwen-Image-Edit-2509 as the world-modeling backbone, and models are trained for 200 steps with DiffusionNFT on 16 NVIDIA A800-80G GPUs.Source: §5.1 · p. 8

5.2 Text-to-Image Generation

• The Decomposed Graded Verifier consistently surpasses the Discrete Binary Verifier and UnifiedReward on text-to-image generation benchmarks.Checkpoint evaluations on 500 held-out prompts also show steadier, more pronounced improvements than binary-reward training.Source: §5.2 · pp. 8–9

5.3 Rule-Governed Generative World Modeling

• Pairwise verification outperforms pointwise verification on game-based tasks, RISEBench, and UniREditBench, with steadier training improvements.Using the ground-truth image as an explicit reference reduces noisy target-state inference and provides more reliable feedback for policy optimization.Source: §5.3 · pp. 9–10, Table 3 · p. 10, Table 4 · p. 10
• Pairwise-trained models generalize to held-out hard, larger-scale, compositional, and visually shifted world-modeling scenarios.The Hard Dataset is excluded from training; results indicate transferable structured-reasoning capabilities beyond task-specific patterns.Source: Figure 4 · p. 9, §5.3 · p. 10, Table 3 · p. 10
• Supervised fine-tuning provides domain-specific execution priors, while subsequent reinforcement learning further develops reasoning ability.Fine-tuning helps the model perform meaningful rule-based transformations rather than leaving inputs unchanged or distorting them, though fully correct solutions remain uncommon.Source: §5.3 · p. 10, Table 3 · p. 10, Table 4 · p. 10
• Structured-reasoning capabilities emerge after training even though the text encoder remains frozen and only DiT parameters are updated.This suggests reasoning ability is not encoded exclusively in the text encoder.Source: §5.3 · p. 11

6 Conclusion

• The Decomposed Graded Verifier, trained with Decoupled RL, is presented as an effective reward model for traditional text-to-image and structured world-modeling tasks.The conclusion reports extensive experimental validation of the verifier and training strategy.Source: §6 · p. 11
• Decoupled RL jointly optimizes sub-question-level binary judgments and global alignment scoring, using precise local feedback to support fine-grained verification and accurate continuous scoring.The work also reports improved out-of-domain multimodal critic capabilities.Source: §6 · p. 11
• Pairwise verification uses ground-truth images as references, replacing reason-then-judge with direct comparison to produce more accurate and reliable reward signals.The conclusion reports extensive experimental validation of the pairwise verification paradigm.Source: §6 · p. 11

A Proof of Ranking Preservation

• The pairwise ranking comparison is conditioned on samples having distinct quality levels and excludes a point-mass quality distribution, where the comparison is undefined.The proof treats this non-degeneracy condition as vacuous to compare otherwise.Source: A Proof of Ranking Preservation · p. 16
• Under GRPO, group-normalized advantages preserve reward ordering, so higher-reward samples receive larger relative update weights.The implication for GRPO is that a reward design preserving true quality ordering gives more reliable relative policy updates.Source: A Proof of Ranking Preservation · pp. 16, 19
• The decomposed graded reward is strictly increasing in the number of satisfied sub-questions, preserving every pairwise comparison between samples of distinct quality.The proof identifies this monotonic relationship as the basis for ranking accuracy of 1.Source: A Proof of Ranking Preservation · p. 17
• The graded reward achieves perfect pairwise ranking accuracy, while binary ranking is strictly worse when two distinct imperfect quality levels occur with positive probability.The graded reward’s accuracy is 1; the strictness condition requires positive probability at two distinct levels below perfect quality.Source: A Proof of Ranking Preservation · pp. 17–18
• Binary verification correctly ranks a distinct-quality pair only when exactly one sample is perfect; it assigns the same reward to all imperfect samples.Thus, two imperfect samples with different true quality levels are tied by the binary reward.Source: A Proof of Ranking Preservation · pp. 17, 19
• Every ranking error from binary verification comes from a pair of imperfect samples with different true quality levels.The proof’s corollary attributes the binary reward’s ranking error precisely to those pairs.Source: A Proof of Ranking Preservation · p. 18

B.1 Training Dynamics across Six Game-Based Tasks

• Intermediate checkpoints were evaluated every 10 training steps on task validation sets, with Gemini3.1-Pro serving as evaluator and ground-truth images as explicit references.Source: §B.1 · p. 19
• Pairwise verification reduces reliance on internal target-state inference by comparing generated outputs directly with ground-truth references, yielding more reliable policy-optimization rewards.Source: §B.1 · p. 19
• Pointwise verification shows more pronounced fluctuations and limited progress across the six tasks.Source: §B.1 · p. 19
• Across six tasks, pairwise verification generally sustains performance improvements and achieves higher final scores than pointwise verification.The training-dynamics evaluation compares validation scores and score gaps across the six tasks.Source: Figure 5 · p. 19, §B.1 · p. 19

B.2 Sub-question-level Verification Accuracy

• Incorrect sub-question judgments can introduce noise into aggregated rewards even when the prompt decomposition is appropriate.Source: §B.2 · p. 19
• Sub-question accuracy was evaluated on 200 held-out binary samples, with three authors and two Ph.D. students annotating 40 samples each to label the full set.Source: §B.2 · p. 19
• The base model, SFT-trained graded verifier, and Decoupled-RL-optimized verifier were compared against human, GPT-5.5, and Gemini-3.1-Pro annotations; Decoupled RL substantially improved accuracy across all three sources.Accuracy was reported separately for each annotation source.Source: Table 5 · p. 20, §B.2 · p. 20

B.3 Evaluation on Qwen-Image-2512

• The evaluation compares Qwen-Image-2512 (20B) with reinforcement-learning variants guided by discrete binary and decomposed graded verifiers, using GenEval and CLIPScore.It complements experiments on Stable Diffusion 3.5 Medium (2.5B) and FLUX.1-dev (12B).Source: §B.3 · p. 20
• 0.82 overall GenEval improves from 0.74 and exceeds the binary-reward counterpart by 0.06; CLIPScore rises from 34.1 to 35.3, versus 34.4 under binary rewards.These results extend evidence for decomposed graded rewards to a larger generation backbone with stronger baseline performance.Source: §B.3 · p. 20, Table 6 · p. 20
• Attribute binding improves from 0.52 to 0.69 and position from 0.38 to 0.51, the largest gains over the original model.The passage suggests graded rewards are particularly useful for compositional requirements involving attribute associations and spatial relations.Source: §B.3 · p. 20

B.4 Evaluation on InternVL3-8B

• The ViVerBench mean score across 16 task categories rises from 0.606 with graded SFT to 0.652 after Decoupled RL on InternVL3-8B.The evaluation also includes the base model and graded SFT variant.Source: §B.4 · p. 20, Table 7 · p. 21

B.5 Sensitivity to the Number of Sub-questions

• On 200 held-out samples, the verifier was prompted to generate at most 10, 11–20, or 21–30 sub-questions to test decomposition granularity.The experiment investigates whether more sub-questions consistently improve scoring or introduce errors.Source: §B.5 · p. 21
• 0.039 mean alignment error is lowest for 11–20 sub-questions, versus 0.083 for at most 10 and 0.072 for 21–30.Lower mean alignment error is better; simply increasing the number of sub-questions does not guarantee better performance.Source: §B.5 · p. 21, Table 8 · p. 21
• Excessive decomposition can repeatedly ask whether the same requirement is satisfied, causing it to be overweighted in the aggregated score.The example prompt is “A red apple is placed on a table.”Source: §B.5 · p. 21
• Excessive decomposition can also introduce criteria unsupported by the prompt, such as whether an apple is a fruit or is round.These irrelevant criteria can distort the aggregated score.Source: §B.5 · p. 21
• Generating more sub-questions can violate the required output format and reduce final-score reliability, underscoring the need for faithful, non-redundant coverage.Source: §B.5 · p. 21

Limitations

• The evaluation covers only rule-governed game environments with uniquely determined ground-truth images, capturing part of real-world complexity and diversity.These controlled tasks were chosen to evaluate pairwise verification.Source: C Limitations and Future Works · p. 21

Future Works

• Future work will explore non-unique ground truths, build larger and more realistic datasets, and scale verifier and generator backbones toward complex real-world scene reasoning and interaction.The long-term goal is multimodal generative models that reason about, simulate, and interact with complex real-world scenes.Source: C Limitations and Future Works · p. 21

D Decomposition Example

• Text-to-image examples decompose varied prompts into individual checks, including failures for instrument-built architecture, a retro-themed photograph, and a requested black backpack.Those checks receive No, Score: 0, while other prompt attributes receive Yes, Score: 1.Source: D Decomposition Example · pp. 23–25

Rule-Governed World Worlding Pairwise

• The pairwise example compares each generated Y-Z grid cell with the reference after projecting the voxel structure along the X-axis.Each grid position becomes a sub-question about whether the generated cell matches its reference counterpart.Source: D Decomposition Example · p. 26
• The pairwise voxel comparison matched 8 out of 9 cells, with a mismatch at (Y = 3, Z = 1).Each cell in the 3 × 3 Y-Z grid was checked against the reference image.Source: D Decomposition Example · p. 26

Rule-Governed World Worlding Pointwise

• The pointwise voxel projection judged 7 sub-questions correct and 2 incorrect, at (Y = 1, Z = 2) and (Y = 3, Z = 1).The inferred target fills both cells, while the generated projection leaves them empty.Source: D Decomposition Example · pp. 27–28

Text-to-Image Verifier Prompt

• The text-to-image verifier prompt requires atomic questions grounded in explicit prompt facts, image-only answers, binary consistency scores, and exclusion of subjective properties.The number of questions should reflect the prompt’s semantic complexity.Source: D Decomposition Example · p. 28

Pairwise evaluation workflow

• The pairwise workflow decomposes game rules into atomic, visually checkable rules, checks each against the reference image, and returns per-rule judgments plus a global score rounded to two decimals.Its output is a JSON object with sub-questions and a final score; each sub-question has an ID, True/False answer, and brief explanation when false.Source: D Decomposition Example · pp. 29–30

Pointwise evaluation workflow

• The pointwise workflow decomposes rules and compares the generated image with the expected state inferred from the initial image and rules, then reports rule judgments and a rounded global score.The final score is intended to reflect adherence to the game rules given the initial image.Source: D Decomposition Example · pp. 30–31

Text-to-image sub-question verifier

• The text-to-image sub-question prompt judges only whether the generated image visually satisfies one requirement, returning True or False and a brief mismatch explanation when false.It requires a JSON object containing only the answer and explanation fields.Source: D Decomposition Example · p. 31

Pairwise sub-question verifier

• The pairwise sub-question prompt compares the generated image with the reference image as the standard, using the initial image and game rules as context for a single requirement.It specifies a True/False judgment and a brief explanation of specific mismatches when the answer is false.Source: D Decomposition Example · p. 32

Pointwise sub-question verifier

• The pointwise sub-question prompt infers the expected final state from the initial image and game rules, then evaluates whether the generated image satisfies one requirement.It uses the same constrained True/False answer and false-case explanation format.Source: D Decomposition Example · p. 32