PaperlayerView exact revision on arXiv

Source-linked AI summary

Open Vision Reasoner: Transferring Linguistic Cognitive Behavior for Visual Reasoning

Yana Wei, Liang Zhao, Jianjian Sun, Kangheng Lin, Jisheng Yin, Jingcheng Hu, Yinmin Zhang, En Yu, Haoran Lv, Zejia Weng, Jia Wang, Chunrui Han, Yuang Peng, Qi Han, Zheng Ge, Xiangyu Zhang, Daxin Jiang, Vishal M. Patel

arXiv:2507.05255v2cs.CVcs.CL

TL;DR

The paper asks how linguistic cognitive behaviors can transfer to MLLMs for advanced visual reasoning. It trains OVR with linguistic cold start followed by large-scale multimodal RL, finding early behavior transfer and selective scaling of useful visual patterns. OVR achieves strong reasoning results across language and multimodal benchmarks, while perception-focused RL shows limited scalability.

  • Problem

    The paper investigates how linguistic cognitive behaviors can transfer to MLLMs for advanced visual reasoning.

    Source: §1 · p. 2, §6 · p. 12

  • Method

    OVR combines large-scale linguistic cold-start fine-tuning with subsequent multimodal reinforcement learning on Qwen2.5-VL-7B.

    Source: §1 · p. 2, §2 · p. 4, §7 · p. 12

  • Results

    OVR shows early behavior transfer, selective RL scaling of effective visual patterns, and strong performance across language and multimodal reasoning benchmarks.

    Source: §1 · pp. 2–3, §4.3 · p. 8

  • Takeaways & Limitations

    The findings support behavior-centered training as a way to study and scale cross-modal reasoning in multimodal models.

    Source: §1 · p. 2, §7 · p. 12

    RL focused solely on perceptual tasks has limited scalability because reward increases without corresponding growth in reasoning complexity.

    Source: §5.3 · p. 11

Abstract

from arXiv · show

The remarkable reasoning capability of large language models (LLMs) stems from cognitive behaviors that emerge through reinforcement with verifiable rewards. This work investigates how to transfer this principle to Multimodal LLMs (MLLMs) to unlock advanced visual reasoning. We introduce a two-stage paradigm built on Qwen2.5-VL-7B: a massive linguistic cold-start fine-tuning, followed by multimodal reinforcement learning (RL) spanning nearly 1,000 steps, surpassing all previous open-source efforts in scale. This pioneering work reveals three fundamental insights: 1) Behavior transfer emerges surprisingly early in cold start due to linguistic mental imagery. 2) Cold start broadly memorizes visual behaviors, while RL critically discerns and scales up effective patterns. 3) Transfer strategically favors high-utility behaviors such as visual reflection. Our resulting model, Open-Vision-Reasoner (OVR), achieves state-of-the-art performance on a suite of reasoning benchmarks, including 95.3% on MATH500, 51.8% on MathVision and 54.6% on MathVerse. We release our model, data, and training dynamics to catalyze the development of more capable, behavior-aligned multimodal reasoners.

1. Introduction

OVR addresses how linguistic cognitive behaviors can transfer to MLLMs for visual reasoning through linguistic cold start followed by multimodal RL. The study reports early transfer, selective scaling of useful behaviors, and strong multimodal performance, while noting a perception-focused RL scalability limit.

Problem and approach

  • OVR uses a two-stage pipeline that first transfers linguistic cognitive patterns through cold start, then activates them with large-scale multimodal RL.The pipeline is built on Qwen2.5-VL-7B and is presented as the paper’s central response to the transfer problem.Source: §1 · pp. 2–3

Results

  • OVR achieves strong performance across language and multimodal reasoning benchmarks, including 95.3% on MATH500, 51.8% on MathVision, and 54.6% on MathVerse.The contribution summary and introduction frame these results as evidence of effective behavior transfer.Source: §1 · pp. 2–3

Findings

  • Visual cognitive behavior transfer emerges early during cold start, apparently supported by linguistic patterns encoding mental imagery.The paper identifies this as a central training-dynamics insight.Source: §1 · p. 2
  • Cold start broadly memorizes visual behaviors, whereas RL discerns and scales effective patterns, favoring high-utility behaviors such as visual reflection.This describes how the two training stages contribute differently to behavior development.Source: §1 · p. 2

Limitations

  • Linguistic cold start introduces perceptual degradation, while multimodal RL recovers the loss; perception-only RL remains limited because reward rises without increasing reasoning complexity.The paper motivates integrating more diverse primitive visual cognitive behaviors and future agentic or multi-turn RL.Source: §1 · pp. 2–3

2. Cognitive Behavior Preliminaries

The paper frames reasoning as structured cognitive behavior and extends language-based patterns into visual reasoning. It defines visual counterparts such as reflection, divide-and-conquer, verification, and goal-driven tracing, illustrated alongside linguistic backtracking.

Linguistic cognitive behaviors

  • Language reasoning includes backtracking, verification, subgoal setting, and backward chaining as structured patterns resembling a textual inner monologue.These behaviors provide the linguistic counterparts for the paper’s visual extensions.Source: §2 · p. 3

Illustrative example

  • A single response can combine visual divide-and-conquer, reflection, goal-driven visual tracing, and linguistic backtracking.Figure 2 presents this multimodal combination as an example of multiple behaviors being triggered together.Source: Figure 2 · p. 4

Visual cognitive behaviors

  • The paper defines visual reflection, divide-and-conquer, visual verification, and goal-driven visual tracing as extensions of linguistic cognitive behaviors.Formal definitions, examples, and linguistic counterparts are provided in Table 1.Source: §2 · p. 4

3. Open-Vision-Reasoner

Open-Vision-Reasoner is built from Qwen2.5-VL-7B using linguistic cold-start fine-tuning followed by multimodal PPO-based RL with verifiable rewards. Its data construction combines curated language-only and multimodal reasoning examples.

Training pipeline

  • OVR follows two sequential stages: linguistic cold start establishes cognitive behaviors, then multimodal RL promotes reasoning generalization across text and image tasks.The RL stage uses the Open-Reasoner-Zero setting and verifiable match rewards.Source: §3.1 · p. 4, §3 · p. 4

RL algorithm

  • The RL stage uses lightweight PPO with GAE, updating the policy from trajectories sampled under an older policy.The objective clips policy-ratio updates, while GAE balances bias and variance in advantage estimation.Source: §3.2 · pp. 4–5

Reward design

  • Rewards are binary and rule-based: outputs receive 1 for an exact boxed-answer match and 0 otherwise.The design evaluates correctness while ignoring formatting or stylistic preferences.Source: §3.2 · p. 5

Evaluation

  • Table 2 compares language reasoning and general benchmarks, marking best and second-best open-source results and identifying reproduced metrics.The caption specifies the comparison conventions rather than reporting individual scores.Source: Table 2 · p. 6

Dataset construction

  • The dataset pipeline collects mathematical, scientific, logical, geometric, and visual reasoning problems, then filters noise, removes reward-incompatible samples, and reweights coverage.Distillation produces approximately 2 million cold-start examples and filtering leaves around 300k multimodal RL examples.Source: §3.3 · pp. 5–6

4. Experiments

OVR uses a two-stage training strategy based on Qwen2.5-VL-7B and achieves strong results across language and multimodal reasoning evaluations.

Experiments

  • The experiments evaluate language reasoning and multimodal capabilities using mathematics, general reasoning, visual reasoning, and chart-understanding benchmarks.Source: §4.2 · p. 7, §4.3 · p. 7

Enhanced Language Reasoning and General Capabilities

  • OVR achieves +4.6% on MMLU and +10.4% on MMLU-Pro over parameter-matched competitors.Source: §4.2 · p. 7
  • OVR surpasses other 7B open-source models by over 10% on average across AIME 2024 and AIME 2025, matching leading 32B models.Source: §4.2 · p. 7

Superior Visual Reasoning Abilities

  • OVR surpasses 50% on MathVision and achieves state-of-the-art results among 7B models on DynaMath and MathVerse.Source: §4.3 · p. 8
  • OVR gains +7.2% on MMMU-Pro over prior state-of-the-art methods, indicating transfer of language-trained reasoning to multimodal tasks.Source: §4.3 · p. 8

5. Discussion

OVR’s training produces convergent improvement across linguistic and multimodal reasoning while revealing distinct dynamics for cognitive behavior transfer and visual perception. Cold start broadly instills behaviors but can impair perception, whereas multimodal RL selectively amplifies useful patterns and recovers perceptual performance.

5.1. Analysis of Training Dynamics

  • Reward and average response length increase from 7k to over 12k during RL without training collapse across more than 20 datasets.Switching to longer context lengths triggers renewed reward growth when token length plateaus or declines.Source: §5.1 · p. 8
  • All eight reasoning benchmarks show convergent growth across text and multimodal domains during training.Performance rises through both cold start and RL, with fluctuations near the benchmarks’ peaks.Source: §5.1 · pp. 8–9, Figure 4 · p. 9

5.2. Multimodal Cognitive Behavior Analysis

  • Visual reflection emerges from the beginning of cold start, indicating early transfer of linguistic mental-imagery patterns into visual reasoning.The analysis tracks this behavior across subsequent cold-start and RL steps.Source: §5.2 · p. 9, Figure 5 · p. 10
  • Cold start initially suppresses then broadly memorizes visual behaviors, while multimodal RL filters and amplifies pivotal patterns for generalization.This division of labor distinguishes broad behavioral acquisition from selective scaling.Source: §5.2 · pp. 9–10
  • Backtracking transfer grows from 2.5% to 17.3%, whereas verification remains near zero across cold start and RL.The authors associate this strategic difference with mental-imagination precursors and the fundamental role of backtracking in visual reasoning.Source: §5.2 · p. 10, Figure 5 · p. 10

5.3. Beyond Behavior: Visual Perception Analysis and Future Work

  • Cold start degrades several perception benchmarks, but long-term multimodal RL recovers performance on tasks such as MMBench and BLINK.The reported degradation is attributed to token-distribution shifts from large-scale linguistic data.Source: §5.3 · pp. 10–11, Table 4 · p. 11
  • On OCR and counting, reward increases without corresponding growth in average response length, revealing limited RL scalability for discriminative perception.The authors connect this dynamic to missing core visual cognitive behaviors and motivate multi-turn or agentic approaches.Source: §5.3 · p. 11

6. Related Work

Related work traces the progression from supervised and preference-based multimodal training toward rule-based reinforcement learning and language-only cold starts. It also identifies a remaining need for fine-grained evidence about the cognitive behaviors underlying multimodal reasoning.

6. Related Work

  • Training-dynamics studies also examine perception tasks such as OCR and counting as part of multimodal reasoning evaluation.These tasks expose differences between reward growth and response-length scaling.Source: Figure 6 · p. 11
  • MLLM research progressed from image captioning and supervised Chain-of-Thought datasets toward increasingly challenging reasoning tasks.Later approaches added explicit reflection, self-correction, and preference-based post-training.Source: §6 · pp. 11–12
  • Recent multimodal RL work increasingly adopts rule-based rewards through task-specific objectives and multimodal thinking datasets.These efforts extend RLVR-inspired methods such as GRPO into multimodal settings.Source: §6 · p. 12
  • Language-only cold starts use verbal reasoning sequences to establish human-like behaviors before subsequent multimodal learning.This line of work is associated with behaviors described as visual aha moments.Source: §6 · p. 12
  • Prior cognitive-behavior research links effective reasoning to verification, backtracking, subgoal setting, and backward chaining, with test-time studies observing performance gains when these patterns are invoked.The multimodal literature has comparatively limited fine-grained analysis of these underlying mechanisms.Source: §6 · p. 12

7. Conclusion

The paper proposes a two-stage framework that transfers cognitive behavior from language to vision through linguistic cold start and large-scale multimodal RL. Open Vision Reasoner demonstrates strong linguistic and perceptual benchmark performance while tracing how visual cognitive behaviors emerge and evolve.

  • The two-stage framework combines linguistic cold start with large-scale multimodal RL to investigate and scale cognitive behavior in multimodal reasoning.Source: §7 · p. 12
  • Open Vision Reasoner is presented as the largest open-source RL practice built on Qwen2.5-VL-7B.Source: §7 · p. 12
  • The study reports strong performance across linguistic and perceptual benchmarks and systematically analyzes visual cognitive behavior across training stages.Source: §7 · p. 12
  • The findings are intended to support future research on cognitively aligned multimodal agents and behavior-centered scaling of vision-language reasoning.Source: §7 · p. 12

A. Cold-Start Data Curation Details

The cold-start SFT dataset is curated through collection, filtering, cleaning, difficulty stratification, and strategic reweighting. This pipeline balances reasoning domains, data quality, difficulty, and cognitive richness.

B. More Implementation Details

The implementation initializes policy and critic networks from the cold-start model and trains them with PPO using separate weights, fixed optimization settings, and strictly on-policy updates.

C. Details for Coginitive Behavior Evaluation

The evaluation section defines cognitive-behavior metrics and uses prompted LLM-based assessment, including a Backtracking prompt example. Behavior Transfer Rate compares visual and linguistic behavior emergence on MathVision mini.

  • Behavior Transfer Rate measures cross-modal transfer as the ratio of visual to linguistic cognitive-behavior emergence rates.Source: §C.1 · p. 25
  • Higher Behavior Transfer Rate values indicate stronger generalization of cognitive behaviors from language to vision.Source: §C.1 · p. 25
  • An LLM-based evaluation prompt is designed for each behavior, with Backtracking shown as an example in Figure 7.Source: §C.2 · p. 25, Figure 7 · p. 25

D. More Case Studies

Case studies illustrate OVR’s cognitive behaviors across multimodal tasks, including visual decomposition, subgoal setting, backtracking, and justified prediction.

Case-study coverage

  • The case studies include examples of OVR’s reasoning processes for complex multimodal tasks, OCR-based document understanding, and mathematical reasoning.The supplied figures identify these task settings as case studies highlighting cognitive behaviors.Source: D. More Case Studies · p. 25, Figure 8 · p. 26, Figure 9 · p. 27

Emoji Quiz

  • OVR decomposes emoji images, explores candidate answers through subgoals, backtracks by cross-verifying hypotheses, and produces a justified prediction.This reasoning sequence is shown in the emoji quiz case study.Source: D. More Case Studies · pp. 25–26, Figure 10 · p. 28

Mental imagery

  • A separate case study highlights mental imagery in DeepSeek-R1 as discussed in the paper’s analysis.The figure caption connects the highlighted behavior to the paper’s treatment of mental imagery.Source: Figure 11 · p. 29

Related papers

Loading 2507.05255v2…