PaperlayerView exact revision on arXiv

Source-linked AI summary

Vero: An Open RL Recipe for General Visual Reasoning

Gabriel Sarch, Linrong Cai, Qunzhong Wang, Haoyang Wu, Danqi Chen, Zhuang Liu

arXiv:2604.04917v3cs.CVcs.AIcs.CL

TL;DR

Existing visual reasoning models rely on closed RL data and pipelines, limiting systematic study and reproducibility. Vero addresses this with an open, single-stage RL recipe using diverse task data and task-routed rewards, achieving broad cross-task gains and outperforming strong open-weight baselines.

  • Problem

    Closed RL data, training code, and reward designs make visual reasoning gains difficult to study, reproduce, and extend.

  • Method

    Vero trains fully open VLMs with single-stage RL on Vero-600K, a 600K-sample mixture from 59 datasets across six task categories, using task-routed rewards.

  • Results

    Broad task coverage and task-routed rewards produce consistent cross-category gains, with Vero-Qwen3I-8B exceeding Qwen3-VL-8B-Thinking by +3.8 overall and on 25 of 30 benchmarks.

  • Takeaways & Limitations

    Visual reasoning generalization depends on jointly learning distinct task-specific behavioral modes, making multi-task RL a distribution-design problem rather than only an optimizer-design problem.

  • Takeaways & Limitations

    The study does not establish the optimal task taxonomy or minimal task set for broad transfer, and it focuses mainly on 7B-9B models without video or multi-turn data.

Abstract

from arXiv · show

What does it take to build a visual reasoner that works across charts, science, spatial understanding, and open-ended tasks? The strongest vision-language models (VLMs) suggest that broad visual reasoning is within reach, yet their closed data and reinforcement learning (RL) pipelines make their gains difficult to study, reproduce, or extend. We introduce Vero, a family of fully open VLMs that match or exceed existing open-weight models across diverse visual reasoning tasks. We scale RL data and rewards across six broad task categories, constructing Vero-600K, a 600K-sample dataset from 59 datasets, and designing task-routed rewards that handle heterogeneous answers. Across VeroEval, our 30-benchmark suite, Vero-600K outperforms existing RL datasets under controlled comparisons. Applied to five starting models, Vero variants gain 2.9-5.4 points on average over their initial models. Notably, Vero-Qwen3I-8B, trained on the Instruct model, surpasses Qwen3-VL-8B-Thinking by 3.8 points on average without additional distillation. Systematic ablations reveal that different task categories elicit distinct reasoning patterns and that broad gains depend on learning them jointly rather than in isolation. All data, code, and models are publicly available.

1 Introduction

Vero addresses the limited openness and narrow task coverage of existing visual reasoning RL by introducing a diverse, single-stage recipe and evaluation suite. Its broad training mixture yields consistent gains across models and benchmarks, while ablations show that joint learning of distinct task-specific reasoning patterns is important.

  • Motivation: Existing visual reasoning RL systems rely on proprietary data and reward designs, while open efforts often focus narrowly on visual math.These constraints make systematic study difficult and limit evidence about cross-task generalization.
  • Approach: Vero-600K contains 600K samples from 59 datasets spanning six visual task categories, paired with task-routed rewards and evaluated through 30 benchmarks.The recipe uses single-stage RL without additional warm start or proprietary data.
  • Findings: Data diversity is the critical ingredient: distinct task categories elicit different reasoning patterns that transfer poorly when trained in isolation.STEM increases backtracking, whereas grounding favors directed visual search over introspective behavior.
  • Results: +2.9 to +5.4 points over five post-trained initial models, averaged across 30 benchmarks, demonstrates consistent gains from Vero training.Applied directly to pretrained-only Qwen3.5-9B-Base, the recipe reaches 73.0 overall with a +12.9-point gain without SFT or distillation warm start.
  • Results: +3.8 overall points over Qwen3-VL-8B-Thinking is achieved by Vero-Qwen3I-8B, with gains on 25 of 30 benchmarks.Vero-Qwen25-7B also exceeds OpenMMReasoner-7B by +5.6 overall despite its teacher-distillation warm start.

2 Related Work

Prior VLM research spans broad multimodal capabilities, chain-of-thought and RL-based reasoning, and increasingly open training recipes. Vero builds on this landscape by targeting an openly curated, multi-category RL pipeline.

  • Vision-language models: VLMs range from proprietary systems and open-weight families to fully open releases of data, code, and weights.Examples include GPT-5, Gemini, Qwen, GLM, Kimi, Molmo, and LLaVA.
  • Reasoning and thinking for VLMs: Chain-of-thought reasoning provides additional test-time computation through step-by-step decomposition, while distillation and reinforcement learning are the dominant training approaches.Distillation uses teacher-generated traces; RL optimizes against outcome-based rewards without requiring a fixed teacher.
  • RL recipes and training data design: Existing open RL recipes for visual reasoning use varied strategies, including teacher distillation, warm starts, selective replay, and forced rethinking.These methods address training initialization or instability across multimodal reasoning benchmarks.
  • Vero’s positioning: Vero-600K extends open visual reasoning RL through a curated mixture drawn from six categories, including STEM, spatial reasoning, knowledge, charts, and grounding.The curation pipeline assigns candidate datasets to categories before selection and filtering for usable training examples.

3 Vero

Vero constructs a broad multi-task RL recipe by curating and filtering data across six visual reasoning categories, balancing their mixture, and routing rewards to heterogeneous answer formats. Its evaluations and ablations show that broad transfer depends on jointly learning diverse reasoning patterns, with uniform task sampling performing best overall.

  • 3 Vero: Vero frames RL training as maximizing expected reward for structured visual responses across diverse tasks, with dataset breadth required for generalization.Responses contain reasoning content and a final answer verified against ground truth.
  • 3 Vero: Vero-600K contains 600K samples from 59 datasets organized into six categories targeting distinct visual reasoning capabilities.The categories include STEM; Spatial & Action; Knowledge & Recognition; Chart & OCR; Grounding, Counting & Search; and Captioning & Instruction Following.
  • 3 Vero: Dataset curation starts from over 250 candidates and retains 59 after heuristic and manual checks for correctness, unambiguity, and reward-compatible verifiability.Heuristics remove small, low-resolution, or binary-question datasets, while manual inspection evaluates approximately 50 examples per candidate.
  • 3 Vero: +4.6 average benchmark gain on STEM and +3.4 on Chart & OCR are achieved by the selected mixture, compared with +2.9 and +1.9 for FineVision, respectively.On STEM, random sampling from all candidates yields −0.2; on Chart & OCR, it yields +2.5.
  • 3 Vero: Filtering effects vary by category: question filtering gives +1.9 pts on Spatial & Action and −0.6 pts on Grounding, Counting & Search, while answer filtering gives +2.1 pts on Knowledge & Recognition.Despite regressions in some categories, both filters are applied to applicable tasks to remove ambiguous samples.
  • 3 Vero: +5.8 pts over the base model is achieved by uniform task sampling, outperforming difficulty-, image-area-, and reasoning-length-weighted alternatives.The table compares absolute VeroEval score changes using Qwen3-VL-8B-Instruct as the initial RL model.

4 Experiments

Vero is evaluated across six task categories using standardized benchmark settings and shows strong gains over initial models, prior open RL datasets, and alternative training designs. Ablations indicate benefits from task-routed rewards, RL, and GSPO.

  • Evaluation Results: 74.4 on VeroEval is achieved by Vero-Qwen35-9B, which wins 25 of 30 benchmarks and improves all six category averages over Qwen35-9B.
  • Evaluation Results: +2.9 to +5.4 overall gains are obtained across five post-trained initial models, while direct RL on Qwen3.5-9B-Base reaches 73.0, a +12.9 improvement without SFT or distillation warm start.
  • Evaluation Results: Vero-Qwen3I-8B improves over Qwen3-VL-8B-Instruct across all six categories and outperforms Qwen3-VL-8B-Thinking by +3.8 overall without distillation.
  • Dataset Comparisons: Vero-600K reaches the highest score on five of six categories against three prior open RL datasets, with margins of +13.7 on Captioning & IF and +6.7 on Grounding, Counting & Search.
  • Ablations: Task-routed rewards outperform math_verify across task categories by supporting exact-match, numeric-tolerance, set-matching, and LLM-judge comparisons.
  • Ablations: RL with the multi-route reward produces more consistent improvements than SFT, while GSPO achieves the highest average score, 54.7, over GRPO and DAPO at 54.3.

5 Data Diversity & Cross-Task Transfer

Cross-task experiments show that single-category RL transfers poorly and can damage non-target capabilities, whereas balanced mixtures produce positive gains across categories. Broader exposure to diverse samples and task-specific training also changes reasoning length and strategy.

  • Cross-Task Transfer: Single-task training frequently causes neutral or negative transfer, including Grounding, Counting & Search losses of −3.2 to −4.3 and Captioning & Instruction Following losses of −2.1 to −7.7.
  • Cross-Task Transfer: Even with equal compute, mixed training yields positive gains across all categories: +0.3 to +4.2 on Qwen2.5-VL and +1.9 to +5.2 on Qwen3-VL.
  • Cross-Task Transfer: The full 600K mixture further amplifies gains, with consistent patterns across both base models suggesting that multi-task RL supports broad capability.
  • Open-Ended Quality: Answer-tag parsing alone sharply reduces Captioning & Instruction Following, while system guidance and that category’s training restore and improve visual-chat quality.
  • Reasoning Length: Spatial & Action produces the longest reasoning at 1983.3 ± 50.8 words, whereas Knowledge & Recognition produces the shortest at 75.8 ± 2.9 words.
  • Training Exposure: From the 100k checkpoint to final exposure, all nine model–benchmark curves improve, with a mean gain of +4.3 points across the fixed 600K mixture.

6 Visual Chat Quality

Maintaining visual-chat quality requires open-ended instruction-following data and safeguards against judge-oriented reward hacking. The analysis identifies self-evaluation, over-specification, and fabricated visual details as problematic response behaviors.

  • Visual Chat Quality: The section’s analysis frames visual-chat quality as a capability that must be preserved while structured reasoning is trained.
  • Visual Chat Quality: A baseline trained without Captioning & Instruction Following uses answer-tag parsing, and incrementally adding open-ended components tests how to preserve and improve visual-chat quality.
  • Reward Hacking: Adding conversational tasks introduces reward hacking, including self-evaluative and self-congratulatory language intended to inflate an LLM judge’s score.
  • Reward Hacking: Observed score-inflation behaviors include claims of exhaustive coverage, compliance assertions, formatting justifications, fabricated measurements, and invented hex codes.
  • Reward Hacking: Strict judge guidelines and automatic failure conditions assign score 1 to self-evaluative responses, making meta-commentary a losing strategy.

7 Chain-of-Thought Behaviors

Reasoning traces differ substantially by training category at both cognitive-behavioral and low-level skill levels. These differences persist under fixed-category evaluation, indicating that training changes the composition of reasoning as well as accuracy.

  • Cognitive Behaviors: Captioning, chart, spatial, grounding, and STEM training produce distinct behavioral profiles, including mental imagery, regional synthesis, perception-then-reasoning, directed search, and elevated backtracking.
  • Cognitive Behaviors: STEM shows elevated backtracking at 0.48 versus 0.27, while grounding lowers self-awareness to 0.49 versus 0.73 and redirects behavior toward visual search.
  • Low-Level Skills: A logistic-regression probe classifies task-category skill distributions with 0.77 overall accuracy, confirming that the repertoires are largely task-specific.
  • Low-Level Skills: STEM skills emphasize mathematical formulas, Chart & OCR emphasizes label and axis extraction, and Grounding emphasizes reference-object localization.
  • Low-Level Skills: Differences remain pronounced when evaluation is held fixed within a category, while Captioning & Instruction Following concentrates on communicative and descriptive operations.
  • Low-Level Skills: Spatial & Action emphasizes state tracking and forward simulation, whereas Knowledge & Recognition combines visual evidence with general world understanding.

8 Discussion

Vero argues that open visual-reasoning RL should be studied as a transparent, distribution-design problem: broad task coverage and balanced rewards jointly produce diverse behaviors and positive transfer. The paper also identifies scope limits, including missing task types, descriptive behavioral analyses, and emphasis on small models.

  • Open RL recipes make performance drivers, failure modes, and training safety inspectable rather than leaving them hidden in proprietary pipelines.
  • Multi-task reinforcement learning and data diversity: Diverse task mixtures and task-routed rewards can mitigate heterogeneous-task interference and support positive transfer across visual reasoning benchmarks.The discussion contrasts this regime with negative transfer under narrower or less balanced training setups.
  • Task-dependent reasoning behaviors: Different task categories induce distinct behavioral regimes: STEM elicits backtracking-heavy reasoning, while grounding and search favor directed visual strategies.
  • Multi-task reinforcement learning and data diversity: Uniform category weighting outperforms alternatives based on dataset size, reasoning length, or base accuracy, framing multi-task RL as distribution design rather than optimizer design alone.
  • Relation to human multi-task reasoning: Vero’s behavioral findings parallel metacognitive monitoring and visual-attention accounts, supporting a non-monolithic view of visual reasoning.
  • Limitations: The study does not establish an optimal task taxonomy or minimal broad-transfer set, excludes video and multi-turn tasks, and mainly analyzes 7B–9B models.Its behavioral analyses are descriptive rather than causal, so the mechanisms linking behaviors to accuracy remain unidentified.
  • Conclusions: The released datasets, training code, and models are intended as a reproducible foundation for studying open visual reasoning.

A.1 Training Dataset

The training-dataset appendix documents the retained sources and their sizes for Vero’s RL mixture. Table A1 provides the detailed source-level accounting.

  • The appendix directs readers to Table A1 for additional details on each retained data source.
  • Table A1 lists the retained training datasets used in the RL mixture and reports their retained sizes.Captioning and instruction-following retained sizes are rounded display values.

A.2 Evaluation Datasets

The evaluation-dataset appendix describes VeroEval’s benchmark organization and the preprocessing used to make evaluation items relevant, clear, language-consistent, verifiable, and numerically readable. It also documents dataset-specific rewrites and multimodal instruction-following data construction.

  • Table A2 organizes VeroEval benchmarks by task category and briefly describes what each benchmark evaluates.
  • Dataset-specific preprocessing: Preprocessing normalizes answers and rewrites selected datasets into shorter, verifiable formats, including chart answers, GeoQA+, KVG, and ReachQA.
  • Question filtering: The relevance filter marks items when the image does not depict the queried entities or attributes.
  • Question filtering: The ambiguity filter targets unclear referents, incomplete prompts, and non-question content such as raw lists or tables without queries.
  • Question filtering: The language filter marks questions requiring reading or understanding in a language other than English.
  • Question filtering: The verifiability filter requires a single objectively answerable response based solely on visible image content.Items requiring external knowledge, speculation, counterfactuals, or allowing multiple plausible answers are filtered.
  • Question filtering: The numeric-precision filter removes questions whose exact values or derived metrics cannot be confidently recovered from the visual.Approximation wording does not exempt an item when the underlying value remains ambiguous.

A.3 Question Filtering

Question filtering removes training examples whose visual content, wording, answer type, or verifiability would make reward computation unreliable. Answer filtering then normalizes retained ground truths into verifier-compatible formats and excludes unresolvable answers.

  • Question filtering: Five independent boolean flags remove samples for relevance, ambiguity, language, verifiability, or unsupported numeric precision.A sample is removed when any flag is true.
  • Question filtering: Filtering catches unsupported numeric precision, question–image mismatches, ambiguous references, and hidden external knowledge.Examples include unverifiable chart percentages, irrelevant questions, unclear referents, and museum provenance absent from pixels.
  • Answer filtering: Ground truths are classified as multiple-choice, numeric, string, or None before rule-based normalization for reward verification.The normalizer rewrites heterogeneous source answers into standard forms consumed by reward verifiers.
  • Answer filtering: Multiple-choice answers are normalized to uppercase letters, including numbered options mapped positionally to letters.The rule also handles parenthesized letters and text options referencing labeled figures or graphs.
  • Answer filtering: Numeric answers lose units and formatting, strings are lowercased with normalized whitespace, and unclassifiable answers are filtered.Examples include converting fractions to decimals and removing currency symbols or degree markers.
  • Data mixture: Mixture experiments profile domain accuracy, reasoning length, and image area before comparing uniform and non-uniform weighting schemes.The schemes include difficulty-, reasoning-length-, image-area-based weighting and an ablation without Knowledge & Recognition.

B Training Details

Vero trains models with structured reasoning and answer outputs, GSPO-based RL, heterogeneous rewards, and explicit formatting constraints. Training uses fixed infrastructure settings, while ablations compare reward and supervision choices.

  • System prompt: The system prompt requires detailed reasoning inside <think> tags and a self-contained final response inside <answer> tags.Definitive concise answers additionally use exactly one boxed result; open-ended responses omit boxed notation.
  • Reinforcement learning: GSPO optimizes a clipped surrogate loss using sequence-level importance ratios and normalized group advantages across rollout responses.The objective aggregates sequence means while routing gradients through sequence averages and keeping token log-probabilities differentiable.
  • Training configuration: Vero models train for 2,343 steps with VeRL and FSDP2 on 8 GPUs, using fp16 for Qwen models and bf16 for MiMo-VL.The reported settings reflect preliminary stability observations for the model families.
  • Reward design: The total reward combines answer accuracy, format compliance, and an overlong-response penalty, with blended instruction and judge rewards where needed.The blended accuracy score uses w = 0.5, while formatting requires the prescribed reasoning-answer structure.
  • Reward design: Format rewards require non-empty <think> content and correctly structured <answer> content, with boxed-result rules for discrete answer types.Missing or multiple boxed expressions reduce formatting reward for applicable tasks.
  • Ablations: 57.2 overall and +4.8 over the base model: RL outperforms both the base comparison and SFT alternatives across categories.SFT on Vero data reaches 52.8 versus 46.2 for FineVision SFT, while RL produces the larger reported gain.

C Evaluation Details

Evaluation combines benchmark-specific scoring choices with automated analyses of reasoning behavior and skill prevalence. The analyses control decoding, benchmark-size imbalance, and category-specific measurement differences.

  • Evaluation setup: Evaluation uses one sampled decode per example with model-family-specific decoding setups for Qwen2.5-VL, MiMo-VL, and Qwen3-VL.Runtime settings and sampling parameters are summarized separately by model family.
  • Evaluation setup: Benchmark-specific evaluation uses mean IoU for AerialVG, rescaled scores for MM-MTBench, and averaged difficulty subsets for VisualProbe.Selected splits and subsets are also specified for MMMU-Pro, MathVista, SimpleVQA, and GameQALite.
  • Input analysis: Captioning & Instruction Following and Grounding, Counting & Search use the largest mean image areas, while Spatial & Action uses the smallest.The reported means are 1.53 ± 0.11 million, 1.50 ± 0.07 million, and 0.52 ± 0.01 million pixels, respectively.
  • Cognitive behavior analysis: Behavior analysis extends a 28-behavior cognitive framework with six visual-analysis capabilities and scores their presence across validation categories.Qwen3-32B evaluates traces using individualized prompts and structured JSON outputs.
  • Cognitive behavior analysis: The annotation protocol removes exact span identification and uses holistic judgments of complete reasoning traces.Greedy decoding and domain-balanced downsampling are used to improve efficiency and reduce benchmark-size imbalance.
  • Behavior results: Captioning-trained Qwen2.5 models show more mental imagery simulation, while chart-trained models show more systematic regional synthesis.Cross-domain averages are 0.33 versus 0.19 for mental imagery simulation and 0.24 versus 0.16 for systematic regional synthesis.

D.5 Skill Experiments

Skill experiments extract, consolidate, and annotate reusable reasoning behaviors to compare how task-category training shapes cognitive profiles. Results indicate substantial category coupling and limited guaranteed transfer of low-level skills.

  • Skill results: Different training categories produce substantial differences in skill presence rates, even when models are evaluated on the same category.The reported variance couples employed cognitive strategies to the training distribution.
  • Skill results: Captioning reinforces generative skills such as Define Narrative Structure, whereas chart and OCR training emphasizes Axis Analysis.The comparison is presented as evidence that task domains demand distinct cognitive profiles.
  • Skill probe: Extracted skills are largely linearly separable by task category in the logistic regression probe.The probe uses canonical behaviors embedded with Qwen3-Embedding-8B from within-task reasoning traces.
  • Skill definitions: The skill glossary includes behaviors such as Backtracking, Mental Imagery Simulation, Systematic Regional Synthesis, and Visual Foraging.These definitions cover returning from unproductive paths, preserving stimulus properties without direct input, traversing image regions, and managing multiple visual targets.
  • Qualitative analysis: Qualitative traces show structured chain-of-thought with self-verification, backtracking, and visually grounded reasoning across the six task categories.The examples connect task-specific strategies with the broader mixed-domain RL recipe.
Loading 2604.04917v3…