Source-linked AI summary
V-Rubrics: Visual Faithfulness via Rubric-Based Reinforcement Learning
Shulin Tian, Minglun Li, Yuhao Dong, Hao Ding, Jiarui Yao, Haiwen Diao, Jingkang Yang, Hongyuan Zhu, Ziwei Liu
TL;DR
Vision-language models can produce plausible responses containing unsupported visual claims, while scalar rewards do not identify which facts or reasoning steps fail. V-Rubrics addresses this gap by training with atomic visual, reasoning, and instruction rubrics plus localized credit; rubric-based GRPO improves over SFT and answer-only GRPO, especially on grounded reasoning tasks.
Problem
Scalar outcome rewards cannot identify which visual facts are grounded, reasoning steps are valid, or instruction constraints are violated in mixed-quality responses.
Method
The method decomposes reference responses into atomic Visual Faithfulness, Reasoning Consistency, and Instruction Following criteria, using component-wise, prefix-localized rubric credit.
Results
Rubric-based GRPO improves over the shared SFT baseline and answer-only GRPO, with the largest gains on knowledge-oriented and visually grounded reasoning benchmarks.
Takeaways & Limitations
Rubrics provide a structured reward abstraction for visual post-training that exposes partial credit and where grounded reasoning succeeds or fails.
Takeaways & Limitations
The approach depends on automatically generated rubric quality and judge-model verification, while prefix localization is only approximate.
Abstract
from arXiv · showhide
Vision-language models can produce fluent answers that are insufficiently grounded in the visual evidence: a single unsupported object, chart value, or intermediate inference can undermine an otherwise plausible response. We argue that this is a credit-assignment failure in multimodal post-training. Scalar outcome rewards indicate whether an answer is acceptable, but do not identify which visual facts are grounded, which reasoning steps are valid, or which instruction constraints are missed. We introduce Visual Rubrics-Based Reinforcement Learning, which decomposes reference responses into atomic propositions and scores generated answers along Visual Faithfulness (VF), Reasoning Consistency (RC), and Instruction Following (IF). The resulting rubric items provide structured partial credit and localize rubric credit when supporting evidence spans are available. We first obtain an SFT checkpoint by fine-tuning Qwen3-VL-8B-Instruct on the public OpenMMReasoner-SFT-874K corpus, adapting OpenMMReasoner's cold-start data recipe. We construct V-Rubrics 50K, a 50,248-example training set from 17 visually grounded sources, by applying rule-based filters before deriving example difficulty from rejection-sampling scores and then annotating every example with Gemini-3-Pro under the same structured prompt and protocol. We train our model based on the same SFT checkpoint using component-wise, prefix-localized rubric credit. Experiments show that our rubricbased GRPO improves over both the shared SFT baseline and answer-only GRPO, with the largest gains on knowledge-oriented and visually grounded reasoning benchmarks. The results show rubrics as a useful reward abstraction for visual post-training.
1 Introduction
V-Rubrics frames visual faithfulness as a multimodal RL credit-assignment problem and addresses it by decomposing responses into grounded rubric items with localized credit. The resulting training resource and rubric-based GRPO improve performance over shared SFT and answer-only GRPO, especially on grounded intermediate reasoning.
- Motivation: VLM responses can contain unsupported visual claims that conventional captioning, instruction-following, and scalar rewards fail to localize.These failures are particularly consequential for charts, documents, diagrams, and dense scenes.
- Proposed framework: Visual Rubrics-Based Reinforcement Learning scores atomic criteria for Visual Faithfulness, Reasoning Consistency, and Instruction Following.The criteria respectively assess image support, inference validity, and compliance with prompt requirements.
- Proposed framework: The framework provides partial credit for grounded content and targeted penalties for unsupported or inconsistent claims.When supporting evidence is alignable, rubric advantages are localized to corresponding response prefixes rather than collapsed into one sequence-level score.
- Training resource: V-Rubrics 50K contains 50,248 examples from 17 visually grounded sources with atomic criteria and importance weights.The resource turns visual grounding into structured supervision for post-training.
- Results: Rubric-based GRPO improves over shared SFT and answer-only GRPO, with the largest gains on tasks requiring grounded intermediate reasoning.Ablations favor the combined component-wise, prefix-localized design over scalar sequence-level rubric aggregation.
2 Related Work
Related work motivates rubrics as intermediate reward units for multimodal reasoning and distinguishes V-Rubrics by grounding decomposed feedback in visual evidence. Its training signal factorizes faithfulness, reasoning consistency, and instruction following while enabling prefix-localized credit.
- Multimodal reasoning and reward granularity: Multimodal RL faces mixed responses that combine correct observations, invalid inferences, and formatting constraints.This motivates reward signals more granular than a single correctness outcome.
- Rubrics as a reward interface: Rubric-based evaluation replaces a single correctness bit with explicit criteria, improving interpretability of automated feedback.Prior text and multimodal judges extend this approach to image-conditioned answers.
- Rubrics as a reward interface: V-Rubrics grounds Visual Faithfulness and Reasoning Consistency criteria in image evidence or licensed inference.Instruction Following criteria complete the visually grounded reward interface.
- From alignment feedback to local credit: Unlike holistic preference or critique signals, V-Rubrics decomposes feedback into propositions for faithfulness, reasoning consistency, and instruction following.This decomposition enables prefix-localized, item-factorized credit.
3 Method
The method represents visual answers with atomic, weighted rubrics and trains with component-wise credit that can be localized to supporting response prefixes. It combines rubric supervision with answer rewards through a pipeline built on a shared supervised checkpoint and the V-Rubrics 50K resource.
- Framework: Visual Rubrics-Based Reinforcement Learning converts fine-grained visual rubrics into item-level scores and prefix-localized token advantages.The pipeline covers problem setup, SFT initialization, V-Rubrics 50K construction, rubric design, and rubric-based RL training.
- Problem Setup: Each reference response is decomposed into atomic criteria labeled Visual Faithfulness, Reasoning Consistency, or Instruction Following, with importance labels and numeric weights.The criteria evaluate image-supported content, logically supported conclusions, and compliance with prompt requirements.
- SFT Initialization and Pipeline: The training setup initializes both the trainable actor and frozen KL reference from the same SFT policy derived from Qwen3-VL-8B-Instruct and OpenMMReasoner-SFT-874K.The shared supervised policy also generates rollouts for rejection sampling before subsequent GRPO runs.
- Dataset Construction: V-Rubrics 50K contains 50,248 examples from 17 visually grounded sources, annotated with atomic VF/RC/IF criteria and importance weights.Examples are normalized, filtered with deterministic rules, and annotated using a structured multimodal protocol; Figure 3 summarizes the resulting distribution.
- Rubric Design Principles: Rubrics are designed to be visually grounded, self-contained, coverage-oriented, and importance-aware, with separate difficulty categories derived from rejection-sampling scores.Criteria cover intermediate visual facts and reasoning steps, while central criteria receive greater weight.
- Rubric-Based Reward: An LLM verifier independently assigns binary satisfaction scores to each criterion, treating PITFALL items as avoidance conditions and confirmed violations as semantic vetoes.Positive criteria provide importance-weighted partial credit after weights are normalized over positive items.
- Rubric-Based Reward: The semantic reward blends rubric feedback with a binary final-answer reward, while a standard binary format reward is also used during training.The answer term anchors task success and rubric items identify response components supporting it.
- Rubrics-Guided RL Training: GRPO standardizes score components across rollouts, broadcasts answer credit across the response, and applies available rubric credit only through corresponding prefixes.Unlocalized items receive sequence-wide item-factorized credit, while items with unanimous rollout scores contribute no gradient.
4 Experiments
Experiments evaluate rubric-based GRPO against shared SFT and answer-level GRPO across general, knowledge-oriented, visual-mathematical, chart, and logic benchmarks. Rubric rewards produce their strongest improvements on reasoning-heavy visual tasks while preserving general VLM performance, with ablations supporting component-wise, prefix-localized credit.
- Evaluation: The evaluation covers 10 benchmark families spanning general VLM ability, knowledge-oriented reasoning, visual math, chart reasoning, and logic.Metrics use standard accuracy and unweighted averages over displayed metrics and splits.
- Main Results: Rubric-based GRPO adds 1.79 points to Overall Avg. over answer-level GRPO on the general/knowledge table, with gains concentrated on MMMU and MMMU-Pro.Scalar GRPO is slightly higher on MMBench-Dev, indicating that rubric gains target multi-step academic reasoning rather than broad capability alone.
- Main Results: Rubric-based GRPO improves visual-reasoning Overall Avg. by 4.00 points over SFT and remains slightly ahead of answer-level GRPO.The strongest gains appear on MathVision, DynaMath, WeMath, and LogicVista, where grounded intermediate perception matters.
- Main Results: The improvement is not uniform: scalar GRPO remains better on MathVerse V/O and CharXiv reasoning.Rubric benefits concentrate where rubric items expose useful partial credit.
- Ablations: Component-wise prefix credit reaches 68.04 Overall Avg., versus 67.74 for scalar rubric aggregation and 66.25 for answer-only GRPO.The 0.30-point difference between rubric variants reflects component-wise standardization and localization jointly, not localization alone.
- Qualitative Analysis: Qualitative examples show rubric training preserving an explicit age-to-birth-year inference and tracing visible graph segments before answering.These corrections illustrate how rubric feedback rewards intermediate visual and logical claims that scalar answer rewards collapse into a final verdict.
6 Limitations
The study’s main limitations concern rubric and judge quality, approximate prefix-credit localization, model-family bias, and research-only deployment scope.
- Automatically generated rubrics and judge verification can encode reference-answer bias, visual ambiguity, or unsupported assumptions.The authors propose larger human-audited evaluations as future work.
- Prefix-credit localization uses fuzzy sentence-to-response matching, so it provides practical local feedback rather than exact token-level supervision.
- Qwen-family judges may introduce family bias when scoring Qwen-family policies.The study calls for testing judge diversity and transfer to other model families.
- The training data remain subject to the licenses and usage terms of their 17 upstream datasets.Users must consult and comply with those terms before using or redistributing corresponding records.
- The trained policy is intended as a research artifact, not as a standalone fact-checker or decision system.
B.1 Expanded Related Work
Related work spans open multimodal reasoning, RLVR, long-chain adaptation, rubric judging, hallucination-aware alignment, and difficulty-aware data composition. V-Rubrics distinguishes itself by making grounded atomic rubric items the explicit reward object during reinforcement learning.
- Open VLMs and RLVR methods provide strong visual foundations and adapt verifiable rewards to multimodal reasoning tasks.
- V-Rubrics evaluates each response through grounded rubric items with interpretable dimensions and importance weights instead of only final-answer or benchmark-level rewards.
- Long-chain and adaptation systems supervise reasoning trajectories, whereas V-Rubrics decomposes reference answers into visually checkable atomic propositions for dense RL credit.
- Rubric-based evaluation and reward modeling address the limitation that a single scalar score can hide which response components succeeded or failed.
- Hallucination-aware methods diagnose or reduce unsupported visual claims through evaluation, preference optimization, and fine-grained feedback.
- V-Rubrics records example-level rejection-sampling scores and deterministic difficulty categories for fixed training-data composition.The reported mixture contains 18,121 hard, 25,306 medium, and 6,821 simple examples.
C V-Rubrics 50K Data and Rubric Construction
V-Rubrics 50K is a 50,248-example resource built from 17 visual reasoning sources through filtered selection, difficulty stratification, and structured atomic rubric annotation across VF, RC, and IF.
- C.1 Source Datasets and Final Inventory: V-Rubrics 50K contains 50,248 examples drawn from 17 canonical training sources spanning charts, documents, mathematics, counting, education, and general visual reasoning.
- C.2 Data Construction and Difficulty Composition: Construction begins with rule-based checks for valid fields and media, non-trivial learning targets, language quality, and identity or strict-content deduplication.
- C.2 Data Construction and Difficulty Composition: Rejection sampling supports example selection and supplies scores from which the dataset’s difficulty composition is derived.
- C.2 Data Construction and Difficulty Composition: The release contains 18,121 hard, 25,306 medium, and 6,821 simple examples, with samples scored 8/8 discarded.
- C.3 Rubric Metadata and Generation Schema: Each reference answer is decomposed into independently checkable atomic propositions tagged as Visual Faithfulness, Reasoning Consistency, or Instruction Following.
- C.3 Rubric Metadata and Generation Schema: Gemini-3-Pro generates every rubric annotation from the source image, instruction, and reference answer under one structured image-conditioned protocol.
- C.3 Rubric Metadata and Generation Schema: Table 4 reports the canonical source inventory, with counts based on the 50,248 records and source shares rounded independently to one decimal place.
- C.3 Rubric Metadata and Generation Schema: The schema uses explicit importance labels and weights, while requiring visual grounding, self-containment, and coverage for reward use.
D Reward, Training, and Decoding Details
The training pipeline starts from a Qwen3-VL-8B supervised checkpoint and compares answer-level and rubric-based GRPO runs from the same initialization, with frozen KL references.
- The SFT checkpoint is obtained by training Qwen3-VL-8B-Instruct on OpenMMReasoner-SFT-874K using an adapted cold-start recipe.
- Each GRPO variant independently initializes its actor and frozen KL reference policy from the resulting πSFT checkpoint.
- Answer-level and rubric-based runs optimize separate actor copies on DV-Rubrics while keeping their respective reference copies fixed.
D.2 GRPO Training Configuration
The GRPO runs share the same core initialization, data, rollout size, and training horizon, while differing in selected batch-size and reward or credit-assignment settings.
- Both variants use the same SFT initialization, V-Rubrics 50K examples, rollout group size, and maximum training horizon.
- The separately reported differences are train and PPO mini-batch sizes, plus the reward and credit-assignment mechanism.
- Table 6 documents the GRPO training configuration for the runs reported in Tables 1 and 2.
D.3 GRPO Objective and Policy Update
The objective samples multiple tokenized responses and updates the policy with token-level advantages under a clipped surrogate, while rubric variants alter how reward credit is assigned.
- The scalar sequence-level variant optimizes expected blended reward over the RL training set.
- For each prompt, the rollout policy samples G tokenized responses, each with a valid response length.
- The policy update uses token-level advantages in a standard clipped surrogate objective.
- The token importance ratio compares the current policy with the pre-update actor snapshot, while a frozen reference policy supplies KL regularization.
- Training uses a low-variance KL term, no entropy bonus, and dual clipping for negative advantages.
- In the component-wise variant, answer-equivalence advantage receives α = 0.5, while positive rubric advantages use the remaining budget and are broadcast to valid tokens.
- The scalar variant aggregates rubric and answer rewards before group standardization, whereas rubric components use separately standardized scored subsets.
D.5 Detailed Reward and Training Formulation
The detailed formulation defines weighted rubric criteria, partial-credit scoring, and heuristic prefix localization, alongside fixed difficulty composition and decoding procedures.
- D.5 Detailed Reward and Training Formulation: Rubric criteria are typed as ESSENTIAL, IMPORTANT, OPTIONAL, or PITFALL and assigned positive or negative weights.
- D.5 Detailed Reward and Training Formulation: Positive criteria grant partial credit, while a confirmed PITFALL violation vetoes answer and positive-rubric credit; transport or parsing failures do not.
- D.5 Detailed Reward and Training Formulation: The fixed training mixture contains 18,121 hard, 25,306 medium, and 6,821 simple examples, with difficulty derived from example-level rs_score.
- D.5 Detailed Reward and Training Formulation: Prefix localization aligns each rubric’s verbatim scoring sentence to the tokenized response using a partial-ratio fuzzy match.
- D.5 Detailed Reward and Training Formulation: Matched rubric advantages apply through the aligned endpoint, while unmatched scored items span the full valid response.
- D.5 Detailed Reward and Training Formulation: The prefix mask is coarse because earlier tokens may receive credit from several later items, while answer equivalence remains sequence-level.
- D.5 Detailed Reward and Training Formulation: Rubric aggregation preserves partial correctness, component-wise standardization retains item-level differences, and prefix masks determine where credit applies.
- D.6 Decoding Settings: Rollouts use G = 12 samples, temperature 1.0, disabled top-p, and an 8,192-token maximum; judges use temperature 0.0 with task-specific output caps and retries.
E Additional Analysis
Additional analyses show that dense rubric credit improves aggregate performance over answer-only and scalar rubric variants, but rubric shaping is not uniformly superior across benchmarks.
- 68.04 with component-wise prefix credit exceeds 67.74 with scalar rubric aggregation and 66.25 with answer-only GRPO.
- The 0.30-point gap between rubric variants reflects component-wise standardization and localization jointly, not localization alone.
- Factorized prefix credit can reduce irrelevant credit or blame across mixed observations, reasoning, formatting, and later mistakes.
- Answer-level GRPO remains stronger on MMBench-Dev, MathVerse V/O, and CharXiv reasoning.
- Rubric shaping is therefore not uniformly aligned with every evaluation metric.