PaperlayerView exact revision on arXiv

Source-linked AI summary

SPLIT-RL: Staged Perception-Language Reasoning Training with Claim-Level Advantages

Raja Kumar, Rajat Koner, Ritwick Chaudhry, Zhuowei Li, Nishant Sankaran, Yifan Xing

arXiv:2610.10889v1cs.CV

TL;DR

Entangled CoT training gives visual and language reasoning the same sequence-level advantage, making capability-specific credit assignment difficult. SPLIT-RL stages the two capabilities and adds claim-level visual feedback; it improves average accuracy over GRPO across six benchmarks, while diagnostic probes show gains in both capabilities.

  • Problem

    Entangled CoT mixes visual and language reasoning, so a single final-answer reward cannot directly optimize either capability in isolation and can penalize correct claims.

    Source: §1 · p. 1

  • Method

    SPLIT-RL trains visual reasoning and then language reasoning in disjoint stages, using the description as a bottleneck and CLA-GRPO for claim-level visual feedback.

    Source: §1 · p. 2

  • Results

    SPLIT-RL improves average accuracy over GRPO by 1.4–6.1% across six visual math and logic benchmarks, and diagnostic probes show gains in both VR and LR.

    Source: §1 · p. 3, §5 · p. 10

  • Takeaways & Limitations

    Staging alone beats entangled-CoT GRPO, while completeness and correctness signals are complementary and training LR before VR forfeits the benefit of staging.

    Source: §5 · p. 10

    SPLIT-RL relies on LLM judges that add training cost; with the vision tower frozen, effectiveness at frontier scale remains open.

    Source: §9.9 · p. 21

Abstract

from arXiv · show

Vision-Language (VL) reasoning requires a model to both extract relevant and accurate information from an image (visual reasoning, VR), and to infer the answer from it (language reasoning, LR). Reinforcement learning with verifiable rewards typically trains both through a single chain-of-thought with a final-answer reward. This gives every CoT token the same sequence-level advantage, failing to distinguish capability specific errors. We propose SPLIT-RL, a staged post-training approach that trains VR and LR in disjoint phases. Because a group's rollouts differ along one capability at a time, the group-relative advantage isolates it, and each phase is optimized using phase-specific reward. We further introduce Claim-Level Advantage (CLA-GRPO), which decomposes VR-phase rollouts into atomic visual claims and provides a fine-grained advantage at claim level based on visual-type group formation. Although trained in two phases, trained policy is evaluated like GRPO model, with a single CoT call at inference time. Under this protocol, SPLIT-RL improves average accuracy over GRPO by 1.4-6.1 points across Qwen3-VL models from 2B to 30B-A3B and InternVL3.5-8B. Evaluating each capability using an oracle based diagnostic shows that answer-only GRPO leaves perception unchanged, whereas SPLIT-RL improves both VR and LR.

1 INTRODUCTION

SPLIT-RL separates visual perception from language reasoning to address coarse credit assignment in entangled chain-of-thought training. It improves benchmark accuracy over GRPO and uses distinct signals for visual-claim correctness and description completeness.

1 INTRODUCTION

  • Entangled CoT mixes visual and language reasoning, so GRPO’s single answer-based advantage cannot isolate either capability and may penalize correct claims.When a rollout’s answer is wrong, correct visual claims can receive the same nonpositive sequence-level advantage as the error.Source: §1 · p. 1
  • With a fixed oracle, answer-only training improves LR by +3.8 but VR by +0.0, while SPLIT-RL improves VR by +1.7 and LR by +6.2.These diagnostic results compare capability-specific changes under the stated oracle protocol.Source: Figure 1 · p. 2, §1 · p. 3
  • SPLIT-RL trains visual reasoning before language reasoning in separate stages, using the description as an information bottleneck and answer correctness to assess its sufficiency.The reasoner receives the query and description without the image; the approach requires no ground-truth descriptions or VR-specific data.Source: §1 · pp. 2–3
  • CLA-GRPO verifies atomic visual claims and assigns advantages relative to claims of the same visual type, giving fine-grained credit for claim correctness.The method addresses coarse credit assignment within the VR description.Source: §1 · pp. 2–3
  • The recall reward complements claim-level correctness by encouraging descriptions to include all query-relevant evidence needed by the language-reasoning stage.Completeness and correctness are addressed with separate signals.Source: §1 · p. 3
  • Across six visual benchmarks, SPLIT-RL improves average accuracy over GRPO by +1.4 to +6.1 points across five backbones, including +2.7 on Qwen3-VL-8B.Evaluation uses the standard single-stage CoT inference setting.Source: §1 · p. 3

2 RELATED WORK

Prior vision-language reasoning work commonly trains an entangled perception-and-reasoning CoT with final-answer correctness. Related approaches add process rewards, curated curricula, or visual tools, while SPLIT-RL stages capability optimization with verifiable rewards.

2 RELATED WORK

  • Across vision-language RLVR approaches, a single entangled CoT interleaves perception and reasoning and is trained using final-answer correctness alone.The cited work situates this as the interface retained across a broad range of methods and tasks.Source: §2 · p. 3
  • Multimodal process-reward methods require step-level annotations, while curriculum approaches depend on curated corpora; visual tools add compute.SPLIT-RL instead stages capability optimization with a verifiable reward attached to each stage.Source: §2 · p. 3
  • SPLIT-RL trains one policy in two stages: VR updates description tokens, then LR updates reasoning and answer tokens from the description without the image.CLA-GRPO verifies atomic claims against the image, groups them by visual type, and assigns claim-level advantages.Source: Figure 2 · p. 4

3 SPLIT-RL

SPLIT-RL factors one policy into image-grounded description and description-conditioned answering. Staged rollouts isolate capability-specific learning signals, while recall and claim-level signals refine VR supervision.

3.1 DISJOINT VISUAL AND LANGUAGE REASONING

  • The same policy first describes query-relevant image content, then answers from only the question and description, making the description an information bottleneck whose answer reward tests its sufficiency.Visual facts omitted or misstated in the description cannot be recovered by the reasoner.Source: §3.1 · p. 4

3.2 STAGED TRAINING WITH CAPABILITY-SPECIFIC ROLLOUTS

  • Stage-1 groups vary descriptions while sharing the reasoner, and Stage-2 groups vary answers while sharing a description, so each group-relative advantage isolates the capability under training.Stage 2 starts from the Stage-1 policy and trains on descriptions its tuned describer produces.Source: §3.2 · p. 5

3.3 COMPLETENESS AND CORRECTNESS SIGNALS FOR THE VR STAGE

  • Because the bottleneck answer reward is indirect and can mistake reasoning failure for perception failure, SPLIT-RL adds separate signals for description completeness and claim correctness.The answer reward is a single rollout-level scalar.Source: §3.3 · p. 5

3.3.1 COMPLETENESS: RECALL REWARD

  • A text-only judge scores whether the description itself states every visual fact needed for the question, and the VR-stage reward pairs this completeness signal with the bottleneck answer reward.The recall judge rejects descriptions that state or derive the answer, preventing the answer from being written into the description as a shortcut.Source: §3.3.1 · p. 5

3.3.2 CORRECTNESS: CLAIM-LEVEL ADVANTAGE

  • CLA-GRPO segments descriptions into atomic claims and judges each claim for image faithfulness, answer relevance, and visual type.Visual types include properties such as color or material, spatial position, and geometric relation.Source: §3.3.2 · p. 6
  • CLA-GRPO groups high-relevance claims by visual type across sibling rollouts, adds claim-specific advantages to reward advantages on the claim’s tokens, and gives zero advantage to low-relevance claims or groups without disagreement.It changes the granularity and targeting of the learning signal, not the optimizer objective.Source: §3.3.2 · pp. 6–7

4 EXPERIMENTS

Experiments compare SPLIT-RL with GRPO across visual reasoning benchmarks and model backbones, then examine qualitative behavior, component contributions, capability-specific gains, and training order.

4.1 EXPERIMENTAL SETUP

  • Experiments cover six visual math and logic benchmarks and five backbones; SPLIT-RL uses the split only in training and answers through one CoT call at inference.The GRPO baseline uses 4096 maximum output tokens, while each SPLIT-RL stage uses 2048; training steps are matched.Source: §4.1 · pp. 7–8

4.2.1 MAIN RESULTS

  • SPLIT-RL improves over GRPO by +1.5 to +6.1 average points across five backbones; gains include +2.1 on InternVL3.5-8B and +4.6 over PEARL on Qwen3-VL-8B.On Qwen3-VL-8B, it also exceeds VLM-CapCurr by +7.1 points.Source: §4.2.1 · p. 8

4.2.2 QUALITATIVE RESULTS

  • On a WeMath example, GRPO misreads a coordinate and repeats “Wait” 49 times, while SPLIT-RL solves the example in three steps.The passage attributes the baseline error to perception rather than logic; later reasoning remains consistent with the misread visual fact.Source: §4.2.2 · p. 8

4.3.1 CONTRIBUTION OF EACH COMPONENT

  • Staging alone improves over the base model by +3.2 points and slightly exceeds entangled-CoT GRPO’s 72.6, despite using only answer-correctness supervision.The LR stage is identical across the component variants.Source: §4.3.1 · p. 8
  • Combining recall reward and CLA-GRPO yields 75.3 average accuracy, a +2.3 gain over VR→LR alone and larger than their separate gains of +0.1 and +0.7.The passage interprets this as evidence that description completeness and correctness are distinct requirements whose signals reinforce one another.Source: §4.3.1 · p. 8

4.3.2 IMPROVEMENT OF EACH CAPABILITY

  • On Qwen3-VL-8B, SPLIT-RL improves visual reasoning by +1.7 and language reasoning by +6.2 over the base model, while GRPO leaves visual reasoning unchanged.The diagnostic uses an oracle for the capability not being measured; at 4B, SPLIT-RL gains +1.4 in VR and +7.6 in LR.Source: §4.3.2 · p. 9

4.3.3 ORDER OF STAGED TRAINING

  • Reversing training order yields 72.1, compared with 75.3 for VR→LR, with the largest declines on MathVision (−8.2) and WeMath (−5.1).The authors attribute the loss to the reasoner learning to compensate for incomplete descriptions before the later VR stage.Source: §4.3.3 · p. 9

5 CONCLUSION

SPLIT-RL separates visual and language reasoning within one policy, with claim-level advantages and recall reward supporting visual descriptions. The paper reports gains over GRPO and ablations on complementary signals and training order.

5 CONCLUSION

  • SPLIT-RL improves over GRPO by 1.4–6.1 % average accuracy across model sizes and backbones.The conclusion reports that staging alone beats entangled-CoT GRPO and that training LR before VR forfeits the benefit of staging.Source: §5 · p. 10
  • CLA-GRPO converts per-claim verification into a group-relative advantage per visual type, while recall reward addresses description completeness.The approach uses final-answer correctness as a perception signal without caption-level labels.Source: §5 · pp. 9–10
  • The ablations identify completeness and correctness as distinct, complementary requirements, and indicate that perception should be established before refining reasoning.Training LR before VR forfeits the benefit of staging.Source: §5 · p. 10

6 AI USE STATEMENT

The authors report using generative AI tools for several work activities, while stating that the core research ideas were developed by the authors.

6 AI USE STATEMENT

  • Generative AI tools supported code writing, manuscript editing, related-work discovery, and scientific-figure generation or adjustment.The authors state that they examined all AI-assisted output and independently confirmed implementation correctness.Source: §6 · p. 10
  • The authors state that AI tools were not used for intellectual discussion, scientific ideation, or development of the core research ideas.Source: §6 · p. 10
  • The authors attribute the conceptual contributions and methodological ideas to themselves.Source: §6 · p. 10

8 ETHICS STATEMENT

The ethics statement limits the evidence to controlled benchmark evaluations and does not establish suitability for high-stakes deployment.

8 ETHICS STATEMENT

  • Experiments use publicly available model backbones and established research benchmarks.Source: §8 · p. 10
  • The study involves neither human subjects nor collection of sensitive or personally identifiable information.Source: §8 · p. 10
  • Controlled benchmark evaluations do not establish suitability for applications where incorrect outputs may have real-world consequences.Source: §8 · p. 10

9 APPENDIX

The appendix reports training and evaluation details, ablations, a worked recall-reward example, and two-pass results. In two-pass evaluation, SPLIT-RL improves over both the base model and GRPO, while two-pass accuracy remains lower than single-pass accuracy.

9.2 IMPLEMENTATION DETAILS

  • Training filters the dataset from 16,195 to 13,862 samples, freezes the vision tower, updates the language model, and evaluates with greedy single-generation inference.The staged phases each run 2 epochs, while GRPO runs 4 epochs.Source: Table 3 · p. 15, §9.2 · p. 15

9.3 FULL ABLATION RESULTS

  • The appendix compares staged-training orders using the same VR rewards and isolates VR-stage reward components while holding the LR stage constant.The tables report per-benchmark accuracies and average-accuracy changes, but these passages do not give the cell values.Source: Table 4 · p. 16, §9.3 · p. 16, Table 5 · p. 17

9.4 WORKED EXAMPLE OF THE RECALL REWARD

  • In the worked example, the description includes the circle, center, points, and 21 cm label but omits the 168° central angle required for the arc-length formula, so recall reward is 0.The recall judge checks whether the description alone contains every fact needed by the question.Source: Figure 6 · p. 16

9.5 TWO-PASS EVALUATION

  • SPLIT-RL improves over the base model by +8.1 points on Qwen3-VL-8B and +10.5 on Qwen3-VL-4B in two-pass evaluation, 3.9 and 8.2 points more than GRPO.Table 7 covers two-phase self-loop evaluation, where the model answers from its own description without the image.Source: §9.5 · p. 16
  • Two-pass accuracy is lower than single-pass accuracy for every model; SPLIT-RL loses −3.2 points on Qwen3-VL-8B and −4.1 on Qwen3-VL-4B, the least among the compared models.The corresponding losses are −4.4 and −8.6 for GRPO, and −5.8 and −9.8 for the base models; single-pass inference is used for the main results because it is cheaper and more accurate.Source: §9.5 · p. 17

9.6 PHASE PROMPTS

The phase prompts separate visual description from answer reasoning: the describer reports question-relevant image content without solving, and the reasoner answers from the description and question alone.

9.6 PHASE PROMPTS

  • The VR prompt asks for faithful, question-relevant visual description without reasoning or an answer; it emphasizes exact labels, numbers, units, formulas, and spatial relationships.It also directs the model to describe visible details rather than inventing absent features or measurements.Source: §9.6 · pp. 17–18
  • The LR prompt gives the reasoner only the question and visual description, directing it to solve step by step without access to the image.It calls for a short, linear attempt and permits refusal only after a specific missing fact prevents a natural solution.Source: §9.6 · pp. 17–19
  • The prompt asks the describer to preserve visual distinctions such as line styles, colors, counts, ordering, chart structure, and table entries.Estimated visual values should be hedged, while legible printed text and numbers should be quoted exactly.Source: §9.6 · p. 18

9.7 JUDGE PROMPTS

The judge prompts distinguish description sufficiency and role from claim accuracy. The recall judge is text-only, while the claim-verification judge sees the image and evaluates claims individually.

9.7 JUDGE PROMPTS

  • CLA-GRPO verification judges each atomic claim against the image for correctness, question relevance, and a single visual-claim type.The type categories include object presence, count, color or material, spatial position, text label, geometric relation, line style, absence, scene context, and other.Source: §9.7 · pp. 19–20
  • The recall judge gives a score of 1 only when a description is both sufficient for answering and purely descriptive, and 0 if either condition fails.It checks sufficiency and role, not whether the description is factually correct; printed visual values are allowed, but calculations or answer reasoning are not.Source: §9.7 · pp. 19–20
  • The prompts require claims to be assessed independently: one claim’s error should not affect another, and vague or image-unverifiable claims are marked incorrect.Correctness is judged against the image, not against the answer to the question.Source: §9.7 · p. 20

9.8 ADDITIONAL QUALITATIVE EXAMPLES

Section 9.8 adds a qualitative example identified as Figure 7.

9.8 ADDITIONAL QUALITATIVE EXAMPLES

  • The section adds a qualitative example in Figure 7.Source: §9.8 · p. 21

9.9 LIMITATIONS AND FUTURE WORK

SPLIT-RL adds training cost through LLM judges, and its visual descriptions still trail a frontier model. With the vision tower frozen, gains over GRPO shrink as backbones improve; frontier-scale effectiveness and harder perception-bound data remain open questions.

9.9 LIMITATIONS AND FUTURE WORK

  • LLM judges for the recall reward and CLA-GRPO add training cost.The descriptions also still trail those of a frontier model, leaving perception as the main source of headroom.Source: §9.9 · p. 21
  • With the vision tower frozen, SPLIT-RL’s gains over GRPO shrink as backbones improve.Its effectiveness at frontier scale remains open.Source: §9.9 · p. 21
  • The need for harder perception-bound data remains open.Source: §9.9 · p. 21
  • Future work could extend the approach to video and to settings with multiple interleaved rounds of perception and reasoning.Source: §9.9 · p. 21
Loading 2610.10889v1…