Perception-R1: Advancing Multimodal Reasoning Capabilities of MLLMs via Visual Perception Reward

1 INTRODUCTION

• Existing RLVR improves reasoning but does not effectively improve multimodal perception, which supplies essential information for subsequent reasoning.The authors report that RLVR-trained models’ perception capabilities are not statistically significantly different from those of their original counterparts.Source: §1 · pp. 2–3
• Answer-accuracy-only RLVR can leave perception errors uncorrected because models may guess correctly despite describing visual details that are not present.The authors note that this can reinforce a flawed reasoning path.Source: §1 · p. 2
• Perception-R1 adds a visual perception reward to RLVR to encourage accurate visual understanding and alleviate sparse perception rewards.The paper presents the additional signal as supporting more effective multimodal reasoning by strengthening the perceptual foundation.Source: §1 · p. 2
• The method extracts textual visual annotations from correct-answer CoT trajectories and uses a judging LLM to score consistency with MLLM responses.The annotations serve as references for reward assignment.Source: §1 · pp. 2–3
• With 1,442 training samples, Perception-R1 achieves the best performance on most multimodal benchmarks, surpassing Vision-R1, which uses 200K training samples.The reported visual annotations reach 96% accuracy by manual examination.Source: §1 · p. 3

2.1 MULTIMODAL LARGE LANGUAGE MODELS

• MLLM development has advanced through vision-language adaptors, visual encoders, and larger, more diverse training datasets.The section describes cross-attention, parallel visual experts, and linear projection layers, and notes that LLaVA training data grew from 753K to 9.36M samples.Source: §2.1 · p. 3

2.2 MULTIMODAL LARGE LANGUAGE MODELS REASONING

• Multimodal reasoning work spans CoT distillation, test-time scaling, process-level rewards, and RLVR, including applications to detection, counting, and math reasoning.Prior work also explored additional rewards and long-CoT cold-start followed by large-scale RL.Source: §2.2 · pp. 3–4
• The reviewed reasoning methods largely overlook multimodal perception, which is essential to complex reasoning and difficult to optimize with sparse RLVR rewards.The section identifies this gap as the motivation for addressing perception in RLVR.Source: §2.2 · p. 4

3.1 PROBLEM FORMULATION

• Given a visual input, textual query, and ground-truth answer, an MLLM generates a token sequence intended to reach that answer.The dataset is represented as samples containing those three components.Source: §3.1 · p. 4

3.2 REINFORCEMENT LEARNING WITH VERIFIABLE REWARDS (RLVR)

• RLVR assigns rewards using ground-truth answers; its reward functions include structured-format and answer-accuracy components.The paper calls the setup accuracy-only RLVR when accuracy reward is the central reasoning signal and format reward only enforces structured output.Source: §3.2 · p. 4
• GRPO samples rollout groups, normalizes rewards to estimate advantages, then optimizes the policy with a clipped objective and KL penalty.The objective uses a fixed reference policy, and the KL coefficient controls the penalty’s impact.Source: §3.2 · pp. 4–5

4.1 ACCURACY-ONLY RLVR FAILS TO IMPROVE MULTIMODAL PERCEPTION IN MLLMS

• On MathVista and MathVerse, perception errors account for 68–78% of analyzed failures, while McNemar tests find no significant perception change after accuracy-only RLVR.For 50 MathVista problems, the reported p-values are 0.22 for Qwen2-VL-7B-IT and 0.69 for Qwen2.5-VL-7B-IT.Source: §4.1 · p. 5

4.2 PERCEPTION-R1

• Perception-R1 adds a visual-perception reward to address accuracy-only RLVR’s sparse perception signal, alongside accuracy and format rewards.The paper notes that answer correctness does not guarantee accurate perception.Source: §4.2 · p. 5

4.2.1 CURATION OF VISUAL ANNOTATIONS

• Problem-relevant visual annotations are extracted from correct-answer CoT trajectories as references for evaluating perception during RLVR training.Each annotation is a textual, atomic visual fact considered critical for solving the problem.Source: §4.2.1 · pp. 5–6

4.2.2 VISUAL PERCEPTION REWARD

• A judging LLM checks whether each annotation appears accurately in the policy response, and the visual-perception reward sums the resulting binary judgments.The visual-enhanced reward combines format, accuracy, visual-perception, and repetition-penalty terms; GRPO optimizes the policy using the combined reward.Source: §4.2.2 · pp. 6–7

5.1 EXPERIMENT SETTINGS

• Training uses 1,442 annotated samples selected from Geometry3K, and evaluation covers four math and four general multimodal benchmarks.Gemini-2.5-Pro generates CoT trajectories; Qwen2.5-32B-IT extracts annotations from correct-answer trajectories.Source: §5.1 · p. 7

5.2 MAIN RESULTS

• Perception-R1 outperforms previous methods on every benchmark except EMMA, improves measured multimodal perception, and reports strong performance using 1,442 training samples.The authors also report average improvement significant at p < 0.01 versus Vision-R1-7B and MM-Eureka-7B, and over 100× less data than Vision-R1 and 10× less than MM-Eureka.Source: §5.2 · p. 8

5.3 ABLATION STUDY

• Removing either the visual-perception reward or repetition penalty lowers accuracy across benchmarks, while SFT with the same trajectories performs worse than the base model on most benchmarks.Visual-reward ablations also outperform accuracy-only RLVR on MathVerse’s Vision-Only subset; direct use of Qwen2.5-VL-32B-IT as reward model does not outperform Perception-R1.Source: §5.3 · pp. 8–9, Table 2 · p. 9

5.4 FURTHER ANALYSIS OF VISUAL PERCEPTION REWARD

• Different nonzero γ values yield similar average performance and outperform omitting the visual reward, while weaker judging LLMs consistently reduce performance.With the 7B judge, MathVerse is 46.1% versus 47.4% for the original model, and MathVision is 24.2% versus 25.1%; its visual reward also rises rapidly and saturates early.Source: §5.4 · p. 9, Figure 3 · p. 9

• McNemar’s test found no statistically significant perception improvement from accuracy-only RLVR compared with the original MLLMs.Source: §6 · p. 10
• Perception-R1 adds a visual perception reward to the standard accuracy reward to encourage accurate visual perception during RLVR training.Source: §6 · p. 10
• Using 1,442 training samples, Perception-R1 achieved the best performance against multiple baselines on most multimodal math and general benchmarks.Source: §6 · p. 10

• The authors identify a clear methodological description, core implementation configurations, and open-source resources as three reproducibility efforts.Source: REPRODUCIBILITY STATEMENT · p. 10
• Section 4.2 describes the visual perception reward and Perception-R1 implementation, while Appendix C.2 provides the annotation and consistency-judging prompts.Source: REPRODUCIBILITY STATEMENT · p. 10
• Appendix C.1 lists training configurations, and the supplementary materials include the dataset and training code; the authors say they will release the dataset, code, and checkpoint upon publication.Source: REPRODUCIBILITY STATEMENT · p. 10

A.1 BENCHMARKS •

• MathVista, MathVerse, MathVision, and WeMath cover mathematical reasoning through visual question answering, varying text–visual proportions, competition problems, and hierarchical knowledge concepts.Source: §A.1 · p. 16
• MMMU and EMMA assess multidisciplinary multimodal tasks, MMMU-Pro adds more robust evaluation settings, and MMStar requires visual information while avoiding data leakage.Source: §A.1 · p. 16

A.2 BASELINES •

• URSA uses SFT and a process reward model, while R1-VL adds step-wise rewards and R1-OneVision combines cold-start initialization with accuracy-only RLVR.Source: §A.2 · pp. 16–17
• OpenVLThinker iterates SFT and RL, VLAA-Thinker uses mixed rewards, SophiaVL-R1 adds a thinking reward model, and MM-Eureka and Vision-R1 use accuracy-only RLVR.Source: §A.2 · p. 17

B.1 DETAILS OF ANALYSIS OF ACCURACY-ONLY RLVR-TRAINED MLLMS

• The analysis compares perception-error contributions to incorrect cases and perception capabilities before and after accuracy-only RLVR training.It trains Qwen2-VL-7B-IT and Qwen2.5-VL-7B-IT on Geometry3K and manually assesses CoT trajectories on geometry-related MathVista and MathVerse subsets.Source: §B.1 · pp. 17–18
• On 50 MathVista problems, Qwen2.5-VL’s visual perception accuracy rose from 44% to 48%, while problem-solving accuracy rose from 60% to 76%.Among correctly solved problems, incorrect visual perception increased from 36.7% to 39.4%.Source: §B.1 · p. 18

B.2 MANUAL EXAMINATION OF GENERATED VISUAL ANNOTATIONS

• Only 4 of 100 manually examined visual annotations had minor perceptual inaccuracies, corresponding to 96% perceptual accuracy.Source: §B.2 · p. 18
• The authors flag possible implicit reasoning distillation from extracted annotations and classify content as visual, reasoning, or text information to examine that risk.Visual information comes only from the image; reasoning information is absent from the image and problem text but obtainable through reasoning; text information comes directly from the problem text.Source: §B.2 · p. 18
• Under a classification that treats even mild inference as reasoning information, visual information made up 82% of Geometry3K annotations versus 25% of GeoQA+ annotations.The authors identify Geometry3K’s larger visual-information share as the reason for choosing it as training data.Source: §B.2 · pp. 18–19

B.3.1 GENERALIZE TO QWEN2-VL MODEL

• Perception-R1-Qwen2 achieved the best performance on most benchmarks except MathVision, with 39.2% versus 30.1% on MathVerse’s Vision Only subset.The authors attribute its sub-optimal MathVision performance to limited training-data diversity.Source: §B.3.1 · p. 19

B.3.2 GENERALIZE TO MULBERRY DATASET

• The Mulberry training set contains 16.8K samples, mainly from IconQA and DVQA, and no geometry data.The pipeline uses Qwen3-VL-235B-A22B-Instruct to generate reasoning trajectories and Qwen3-Next-80B-A3B-Instruct to extract visual annotations.Source: §B.3.2 · pp. 19–20
• The authors evaluate the Mulberry-trained model in Table 9 against results from applying the method to Qwen2-VL-7B-IT in Table 8.The supplied passages identify the tables’ scope but do not provide their cell values.Source: Table 8 · p. 20, Table 9 · p. 20
• 2.1 points on average across benchmarks separates Perception-R1-Mulberry-7B from standard GRPO.The authors suggest the Mulberry data’s lack of math reasoning content, especially geometry, may explain why Perception-R1-7B performs better on math benchmarks.Source: §B.3.2 · p. 20, Table 9 · p. 20

B.4 ROBUSTNESS ANALYSIS OF PERCEPTION-R1

• The analysis tests two robustness factors: visual-perception reward correctness and γ, which controls that reward’s influence on the final reward.Judgment noise is simulated by flipping Qwen2.5-32B-Instruct judgments at fixed proportions of 10% and 20%.Source: §B.4 · p. 20
• With 20% of visual-perception judgments corrupted, average performance still surpasses GRPO.Degradation mainly comes from MathVista and MathVerse, while corrupted-annotation models perform on par with Perception-R1-7B on MMMU and MMMU-Pro.Source: §B.4 · p. 20, Table 10 · p. 21
• Models trained with different γ values except 0.0 have very similar average performance, and all significantly surpass standard GRPO.Table 11 presents per-benchmark results for the different γ settings.Source: §B.4 · p. 20, Table 11 · p. 21

B.5 PERFORMANCE ON VISION-ONLY BENCHMARKS

• Perception-R1 improves average performance by 2.6 and Perception-R1-Qwen2 by 6.3 on the vision-only subsets of MathVerse and MMMU-Pro, surpassing standard GRPO and the previous SOTA method.Table 12 compares the methods on these two vision-only subsets.Source: §B.5 · p. 21, Table 12 · p. 22

B.6 ANALYSIS OF USING QWEN2.5-VL-32B-IT AS REWARD MODEL

• With Qwen2.5-VL-32B-IT as reward model, visual-perception reward saturates around 100 training steps, while accuracy reward falls below Perception-R1’s, indicating reward hacking that undermines reasoning performance.Figure 5 compares accuracy and visual-perception rewards for Perception-R1 and the reward-model variant.Source: §B.6 · p. 21, Figure 5 · p. 22

B.7 COMPUTATIONAL COSTS COMPARISON

• Perception-R1 takes about 16 hours on 16 A800 GPUs and costs about 167.4 A800-Hours; judgment increases per-step time by 44.5% over standard GRPO, but the method uses substantially less data.Judgment takes 47.5 seconds of each 154.3-second RL step; the comparison categorizes costs as data preparation and training time.Source: §B.7 · pp. 21, 23, Table 13 · p. 23

B.8 CASE STUDIES

• Qualitative cases show accuracy-only RLVR models retaining perception errors, while Perception-R1 alleviates such failures and exhibits a distinct describe-then-solve behavior.The cases associate more accurate visual interpretation with correct solutions.Source: §B.8 · pp. 23–24

C.1 EXPERIMENTAL SETTINGS

• Experiments use Qwen2.5-VL-7B-IT with EasyR1 and Qwen2.5-32B-IT as judging LLM, training for 25 epochs with global batch size 128.The learning rate is 1e-6, warmup ratio 0.05, and reward coefficients α = 0.1, β = 0.9, and γ = 0.7; training takes about 16 hours on 16 NVIDIA-A800-80G GPUs.Source: §C.1 · pp. 28–29

C.2 PROMPTS

• The paper provides prompts for RLVR training, extracting visual annotations from CoT trajectories, and judging consistency between annotations and policy rollouts.The extraction prompt targets diagram-related visual information; the judging prompt checks whether each key item appears in the response.Source: §C.2 · p. 29, Figure 6 · p. 29, Figure 7 · p. 30

D BROADER IMPACTS

• The authors cite education as a possible positive impact of enhanced multimodal perception and reasoning, while warning that these capabilities must be properly harnessed to avoid societal risks.They suggest future research attend more to multimodal perception and perception-oriented enhancements in RLVR.Source: D BROADER IMPACTS · p. 29

E THE USAGE OF LLMS STATEMENTS

• LLMs and MLLMs support data collection, visual-annotation extraction, consistency judgments during training, and manuscript language polishing.Gemini-2.5-Pro generates Geometry3K CoT trajectories, a text-only LLM extracts annotations, and GPT-4o assists with polishing.Source: E THE USAGE OF LLMS STATEMENTS · p. 29