PaperlayerView exact revision on arXiv

Source-linked AI summary

UniCAR-RL: Seeing Better before Thinking Deeper in Visual Mathematics

Yuzhe Li, Hao Yan, Hao Wang, Xingchen Liu, Ya-Qi Yu, Jihao Wu, Minghui Liao, Wei Chen, Yuliang Liu

arXiv:2609.13849v1cs.AI

TL;DR

Fine-grained visual misperception can undermine MLLM mathematical reasoning, and existing RL rewards do not distinguish perception errors from logical ones. UniCAR-RL separates perception and reasoning optimization through three branches using raw short-answer data. It reports gains across mathematical benchmarks and generalization across architectures and sizes.

  • Problem

    Fine-grained visual misperception can seed false premises in mathematical reasoning, while sparse RL rewards do not distinguish perception errors from logical missteps.

    Source: §1 · p. 1

  • Method

    UniCAR-RL uses verifier-guided caption optimization, reasoning from verified gold descriptions, and end-to-end QA-RL to optimize perception and reasoning from raw short answers.

    Source: §1 · pp. 1–2

  • Results

    UniCAR-RL reports comprehensive gains across diverse mathematical benchmarks and robust generalization across model architectures and sizes.

    Source: §1 · p. 2, §5 · p. 9

  • Takeaways & Limitations

    The reported gains are said to generalize to broad multimodal tasks, particularly for smaller models and earlier architectures.

    Source: §5 · p. 9

    The three-branch framework increases training cost and has been validated primarily on complex mathematics and a limited set of general visual tasks.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Multimodal Large Language Models (MLLMs) often struggle with complex mathematical visual reasoning primarily due to a lack of fine-grained perception, causing initial visual hallucinations to directly trigger cascading reasoning failures. In traditional end-to-end reinforcement learning (RL), sparse rewards fail to decouple perceptual hallucinations from logical missteps, hindering targeted perception optimization. Alternatively, fine-tuning with perception-enhanced CoT data incurs high costs and hallucinations. In this paper, we address these challenges by proposing UniCAR-RL, an annotation-free RL framework. By explicitly decoupling the optimization of perception and reasoning during the training process, it achieves isolation and optimization of both capabilities. Specifically, UniCAR-RL consists of three synergistic branches: 1) a Caption-RL branch that optimizes perception capabilities through verifier-guided reasoning validation; 2) a Reasoning-RL branch that performs logical reasoning based on a gold image description to halt cascading errors; 3) a QA-RL branch that retains native end-to-end alignment to ensure robust question-answering performance. Experiments show that UniCAR-RL substantially improves MLLMs' mathematical and visual reasoning using only raw short-answer data. Furthermore, it demonstrates strong generalization across diverse architectures and scales.

1. Introduction

Fine-grained visual misperception can seed false premises that derail mathematical reasoning, while sparse RL rewards do not distinguish perception errors from logical errors. UniCAR-RL separates these signals through three synergistic branches and reports gains across mathematical benchmarks and model architectures and sizes.

1. Introduction

  • Misreading critical, microscopic visual conditions can make otherwise rigorous mathematical reasoning rely on hallucinated premises.The paper identifies fine-grained perception, rather than insufficient reasoning skill, as the primary bottleneck in abstract visual reasoning tasks.Source: §1 · p. 1
  • Sparse global RL rewards cannot distinguish visual misperception from logical errors, while perception-enhanced CoT methods rely on costly expert annotations and may inherit teacher hallucinations.These constraints make targeted perception optimization difficult with existing approaches.Source: §1 · p. 1
  • UniCAR-RL uses an image-description process and an advanced verifier to disentangle perceptual evaluation from end-to-end black-box training.Caption-RL rewards descriptions when a verifier can deduce the ground-truth answer from them, providing direct perception feedback without reasoning-error interference.Source: §1 · pp. 1–2
  • Reasoning-RL reasons from a verified gold caption, while QA-RL jointly optimizes perception and reasoning end-to-end and retains question-answering proficiency.Together, the branches use raw short answers to improve perception while preserving reasoning optimization.Source: §1 · p. 2
  • UniCAR-RL reports comprehensive gains across diverse mathematical benchmarks, with performance comparable to close-source proprietary models at a fraction of the training cost.The framework also reports consistent, significant improvements across architectures and sizes.Source: §1 · p. 2
  • UniCAR-RL reports consistent, significant performance improvements across architectures and sizes, with optimization efficacy showing limited sensitivity to verifier capabilities.The framework's performance gains are reported as comparable to close-source proprietary models at a fraction of the training cost.Source: §1 · p. 2

2. Related Work

Related work on multimodal mathematical reasoning includes cold-start and zero-RL approaches, while visual-perception methods use perception-enhanced trajectories or perception-guided rewards. Other approaches seek to reduce reliance on distilled perception data.

RL for Mathematical Reasoning

  • Cold-start RL bootstraps policies with curated traces or process supervision, whereas zero-RL directly optimizes MLLMs with rule-based rewards.Examples include We-Math 2.0 [20], Vision-R1 [21], URSA [22], MM-Eureka [23], Shuffle-R1 [4], R1-VL [24], and VL-Cogito [25].Source: §2 · pp. 2–3

Visual Perception Enhancement in MLLMs

  • Perception-augmented trajectory methods distill perception-enhanced CoT data, while perception-reward-guided RL incorporates perceptual signals into reward design.Examples include CodePercept [15], GeoCode [27], MathFlow [26], ViRC [28], Geoint-R1 [29], and methods using annotation consistency, visual knowledge internalization, or caption utility [30] [31] [32].Source: §2 · p. 3
  • VPPO [33], Vision-SR1, PAPO, and NoisyRollout [35] reduce reliance on distilled perception data using token-focused updates, self-verification, perturbations, or noisy visual rollouts.Source: §2 · p. 3

3. Method

UniCAR-RL separates visual perception, logical reasoning, and end-to-end question answering into jointly updated branches. It uses task sufficiency to evaluate descriptions and a selected description to train reasoning without visual input.

3.1. Overall Framework

  • UniCAR-RL jointly updates three branches with shared policy parameters: question-agnostic perception, text-only reasoning, and native end-to-end question answering.Visual-description sufficiency is measured by whether a text-only solver can derive the correct answer from the description alone.Source: §3.1 · p. 3, Figure 2 · p. 4

3.2. Caption-RL Branch

  • Caption-RL omits the question, generates exhaustive image descriptions, and rewards descriptions from a text-only verifier’s ability to recover the correct answer.The verifier reward is a proxy for caption sufficiency and may contain verifier noise; missing a critical visual detail can cause verification failure.Source: §3.2 · p. 3

3.3. Reasoning-RL Branch

  • Reasoning-RL selects the highest-reward caption and trains the model to reason from that description and the question without image input.The selected caption is drawn from the policy’s own rollouts, and improving captions provides reasoning with progressively richer textual inputs.Source: §3.3 · p. 4

3.4. QA-RL Branch

  • QA-RL trains on the image and question in the test-time format, integrating specialized perception and reasoning while preserving end-to-end task alignment.It runs alongside both specialized branches in every training iteration.Source: §3.4 · pp. 4–5

3.5. Unified GRPO Optimization

  • UniCAR-RL jointly optimizes all three branch objectives in one backward pass, applying specialized gradient signals to shared parameters at each update.Advantages are normalized independently within each rollout group to prevent any branch from dominating the gradient.Source: §3.5 · p. 5

4. Experiments

UniCAR-RL improves mathematical visual reasoning across model scales and architectures, while also improving visual perception and general-task performance. Its gains come with additional training cost, and the experiments examine the contributions of its branches and components.

4.1. Experimental Setup

  • The evaluation uses 30k MMRL30k samples for training and tests on four mathematical benchmarks, with ChartQA and HallusionBench assessing general capabilities.The experiments use Qwen2.5-VL and Qwen3-VL backbones and full-parameter fine-tuning.Source: §4.1 · p. 5

4.2. Main Results

  • UniCAR-RL consistently improves visual mathematical reasoning across backbones and scales, with state-of-the-art average performance among comparable open-source models and competitiveness with proprietary models.Table 1 reports results across WeMath, MathVerse, MathVista, and MathVision.Source: §4.2 · p. 5, Table 1 · p. 6
  • Smaller models and lower-performing baselines tend to gain more, while a stronger verifier provides more precise perception rewards and raises the reasoning-performance upper bound.The reported gains generalize across architectures and parameter scales.Source: §4.2 · p. 5, §4.5 · p. 7

4.3. Visual Perception Evaluation

  • With Gemini 3 Pro as the unified verifier, description-based evaluation finds significant perception gains from UniCAR-RL, whereas standard GRPO brings negligible improvement.Verifier accuracy on problems solved from generated image descriptions is used as the perception measure.Source: §4.3 · p. 5, Table 3 · p. 7

4.4. General Tasks Evaluation

  • Performance gains on ChartQA and HallusionBench across model scales indicate that UniCAR-RL’s perception enhancements transfer beyond mathematical reasoning to diverse visual tasks.Table 4 reports the general multimodal benchmark comparison.Source: §4.4 · p. 6, Table 4 · p. 7

4.5. Ablation Study

  • Removing any branch degrades performance; decoupled perception-and-reasoning optimization yields a 3.1% improvement versus 1.3% for vanilla GRPO.Component ablations also report enhancements across base models and parameter scales, with relatively limited marginal gains from stronger verifiers.Source: §4.5 · pp. 6–7, Table 2 · p. 7, Table 5 · p. 8

4.6. Qualitative Analysis

  • Qualitative examples attribute baseline failures to perception and reasoning errors; UniCAR-RL is described as targeting both, unlike GRPO’s reported reasoning-error correction.The multi-branch approach also retains RL’s error-correction advantages while addressing visual misjudgments.Source: §4.6 · p. 7, Figure 3 · p. 8

4.7. Training Efficiency Analysis

  • A UniCAR-RL iteration takes about 3.75× as long as a vanilla GRPO step, but fewer observed iterations yield an overall training-time ratio of about 1.75×.The reported runs use about 70 iterations for UniCAR-RL versus about 150 steps for vanilla GRPO; Caption-RL accounts for most added per-step overhead.Source: §4.7 · p. 8, Table 6 · p. 8

5. Conclusion

UniCAR-RL decouples perception and reasoning optimization without data distillation, using verifier feedback and gold-description-based reasoning. Experiments report generalization across architectures and sizes, including broader multimodal tasks.

  • UniCAR-RL addresses visual perception bottlenecks in complex mathematical reasoning through annotation-free reinforcement learning without data distillation.Verifier feedback and gold-description-based reasoning provide distinct optimization signals for perception and logical reasoning.Source: §5 · p. 8
  • The framework jointly optimizes fine-grained perception and logical reasoning capabilities.Its three synergistic branches are Caption-RL, Reasoning-RL, and QA-RL.Source: §5 · p. 8
  • Experiments report robust generalization across model architectures and sizes, with enhancements also extending to broad multimodal tasks.The conclusion highlights particularly strong gains for small models and earlier architectures.Source: §5 · p. 9

Limitations

The framework incurs higher training costs and has so far been validated mainly on complex mathematics and a limited set of general visual tasks.

  • Jointly optimizing all three branches increases training cost.The paper discusses this overhead in its training-efficiency analysis.Source: Limitations · p. 9
  • Validation focuses primarily on complex mathematical tasks and a limited set of general visual tasks.Source: Limitations · p. 9
  • Applying UniCAR-RL to a broader range of multimodal scenarios remains future work.Source: Limitations · p. 9

A. Visualization of the UniCAR-RL Training Workflow

A training iteration runs three sequential branches: verifier-scored image descriptions, reasoning from the selected gold caption, and end-to-end question answering. The branches share policy parameters and are jointly optimized.

C. Related Work

The related-work comparison positions UniCAR-RL as combining Zero-RL, no distilled supervision, and explicit visual-perception optimization. The supplementary materials also describe its caption-only reasoning prompt and an additional-results table.

C. Related Work

  • UniCAR-RL combines Zero-RL, avoids distilled supervision, and explicitly optimizes visual perception, unlike existing methods that typically address only part of these aspects.The comparison covers training paradigm, use of teacher-generated or synthetic reasoning supervision, and explicit perception enhancement.Source: C. Related Work · p. 13
  • The Reasoning-RL prompt withholds the image so the model reasons purely from the caption.Source: Figure 7 · p. 13
  • Table 7 compares methods by training paradigm, reliance on distilled reasoning traces, and explicit enhancement of visual perception or grounding.Source: Table 7 · p. 13
  • Table 8 reports additional perception results on ChartQA, HallBench, V*Bench, and HR-Bench, using Gemini 3 Pro as the default training verifier.Source: Table 8 · p. 14

D. Additional Results on General Benchmarks

UniCAR-RL is evaluated on additional visual benchmarks and model configurations beyond the previously reported ChartQA and HallusionBench results. The authors report consistent improvements across these additional tasks and architectures.

D. Additional Results on General Benchmarks

E. Prompts for General Benchmarks

General multimodal benchmarks share a chat template that requests internal reasoning and a boxed final answer. Benchmark-specific questions and evaluation protocols vary by task.

E. Prompts for General Benchmarks

Loading 2609.13849v1…