PaperlayerView exact revision on arXiv

Source-linked AI summary

Perceive-to-Reason: Decoupling Perception and Reasoning for Fine-Grained Visual Reasoning

Hongxing Li, Xiufeng Huang, Dingming Li, Wenjing Jiang, Zixuan Wang, Haolei Xu, Hanrong Zhang, Haiwen Hong, Longtao Huang, Hui Xue, Weiming Lu, Jun Xiao, Yueting Zhuang, Yongliang Shen

arXiv:2607.01191v1cs.CV

TL;DR

Fine-grained visual reasoning requires models to locate subtle evidence and reason over it, but existing methods do not explicitly separate these processes. P2R decouples perception from reasoning and consistently improves performance across scales and broader multimodal tasks, including 93.2% on V-Star for P2R-4B.

  • Problem

    Fine-grained visual reasoning requires locating subtle question-relevant evidence in high-resolution images and reasoning over it, while existing approaches do not explicitly separate these processes.

  • Method

    P2R explicitly decomposes reasoning into Perceiver-based evidence localization followed by Reasoner-based answering, with PRA-GRPO aligning training to the two-stage formulation.

  • Results

    P2R consistently improves performance across model scales and broader multimodal tasks, with P2R-4B achieving 93.2% on V-Star.

  • Takeaways & Limitations

    Explicitly decoupling perception from reasoning provides an effective framework for fine-grained visual reasoning across challenging high-resolution and broader multimodal tasks.

  • Takeaways & Limitations

    P2R adds inference cost, while PRA-GRPO's scaling behavior and broader generalization to interactive or long-horizon settings remain unclear.

Abstract

from arXiv · show

Fine-grained visual reasoning remains challenging for vision-language models, especially when small but critical visual cues are buried in high-resolution images. Existing approaches rely on repeated cropping or test-time visual search to introduce local evidence, but they typically do not explicitly distinguish perception from reasoning. In this paper, we propose Perceive-to-Reason (P2R), a unified framework that formulates fine-grained visual reasoning as a two-stage process: the model first localizes question-relevant evidence as a Perceiver, and then answers the question as a Reasoner based on the annotated image and cropped regions. To better align training with this decoupled formulation, we further introduce Perception-Reasoning Alternating GRPO (PRA-GRPO), a role-aware reinforcement learning strategy that alternates between perception-focused and reasoning-focused updates using only final-answer supervision. Built on top of Qwen3-VL-Instruct-2B/4B/8B, P2R consistently improves performance across model scales. In particular, P2R-4B achieves 93.2% on V-Star, 81.9% on HR-Bench-4K, and 80.5% on HR-Bench-8K, substantially outperforming its corresponding backbone. Further experiments show that the benefits of P2R extend beyond high-resolution benchmarks to broader multimodal reasoning tasks. These results suggest that explicitly decoupling perception from reasoning provides an effective framework for fine-grained visual reasoning.

1 Introduction

Fine-grained visual reasoning requires both locating subtle, question-relevant evidence and reasoning over it, but existing methods often entangle these stages. P2R explicitly decouples perception from reasoning and trains this process with PRA-GRPO using only final-answer supervision.

  • Motivation: Fine-grained visual reasoning requires locating subtle question-relevant evidence in high-resolution images and reasoning over it.The task involves determining both where to look and how to reason.
  • Motivation: 81.7% to 90.6%: Qwen3-VL-Instruct-4B improves on V-Star when given oracle bounding boxes and cropped regions.This diagnostic indicates that many errors stem from failing to localize the right visual evidence.
  • P2R: P2R first localizes question-relevant evidence as a Perceiver, then answers as a Reasoner using the annotated image and cropped regions.Evidence localization becomes an explicit intermediate step rather than an implicit byproduct of answer generation.
  • PRA-GRPO: PRA-GRPO alternates perception-focused and reasoning-focused optimization while keeping the other role fixed, using only final-answer supervision without bounding box annotations.This role-aware strategy makes correctness more attributable to the active stage.
  • Results: 93.2% on V-Star, 81.9% on HR-Bench-4K, and 80.5% on HR-Bench-8K: P2R-4B substantially outperforms its corresponding backbone.P2R consistently improves over Qwen3-VL-Instruct base models across scales, with benefits extending to broader multimodal reasoning tasks.

2 Related Work

Fine-grained visual reasoning challenges VLMs to identify subtle visual evidence and reason over it. Existing work localizes relevant regions through iterative zooming or visual tools, while reinforcement learning has improved visual reasoning in VLMs.

  • Fine-Grained Visual Reasoning: Fine-grained visual reasoning requires identifying subtle visual evidence and reasoning over it, but remains challenging for current VLMs.
  • Fine-Grained Visual Reasoning: Existing methods mainly address this challenge by localizing key regions through iterative zooming or visual tools for interleaved visual-textual reasoning.DeepEyes uses reinforcement learning to improve visual tool use.
  • Reinforcement Learning in VLMs: Reinforcement learning has recently been extended from LLMs to VLMs, producing notable progress in visual reasoning.Vision-R1 and MM-Eureka are representative works showing that RL can significantly improve VLM reasoning capabilities.

3 Methodology

P2R decouples fine-grained visual reasoning into perception and reasoning stages within a shared VLM, explicitly localizing evidence before generating an answer. PRA-GRPO aligns training with this structure by alternately optimizing the Perceiver and Reasoner using final-answer correctness.

  • Two-Stage Inference: P2R uses a shared-parameter VLM in two consecutive stages, with role-conditioned policies for the Perceiver and Reasoner.The perception and reasoning stages are denoted by πp(·; θ) and πr(·; θ), respectively.
  • Two-Stage Inference: The Perceiver predicts question-relevant bounding boxes to make visual evidence localization an explicit intermediate decision.The boxes specify one or more rectangular image regions.
  • Two-Stage Inference: The predicted boxes produce an annotated image and cropped evidence images, which the Reasoner uses to generate the final answer.The annotated image overlays boxes on the original image, while cropping extracts the corresponding local regions.
  • Role-Aware Alternating Optimization: PRA-GRPO alternates Perceiver and Reasoner updates while freezing the other role, making final-answer correctness more attributable to each role.Perception evaluates evidence through a fixed Reasoner, whereas reasoning optimizes answer generation conditioned on evidence from a fixed Perceiver.
  • Role-Aware Alternating Optimization: PRA-GRPO uses only final-answer supervision, avoiding ground-truth bounding boxes and task-specific dense rewards.Both roles remain instantiated by the same underlying VLM with shared parameters θ.

4 Experiments

Experiments show that P2R consistently improves Qwen3-VL-Instruct across high-resolution and broader multimodal benchmarks, with gains from both P2R inference and alternating PRA-GRPO training. Additional analyses indicate improved localization, cross-role transfer, deployment-friendly shared parameters, and generalization to grounding tasks.

  • High-Resolution Benchmarks: P2R consistently outperforms corresponding Qwen3-VL-Instruct baselines across V-Star, HR-Bench-4K, and HR-Bench-8K at all model scales.Averaged over the three benchmarks, P2R-2B, P2R-4B, and P2R-8B improve by 8.1%, 11.0%, and 9.7%, respectively.
  • General Perception and Reasoning Benchmark: P2R improves Qwen3-VL-Instruct on MME-RealWorld-Lite by 4.0%, 7.1%, and 7.0% for the 2B, 4B, and 8B models, respectively.The improvements span nearly all subtasks rather than concentrating in a few categories.
  • Effect of P2R Inference: 93.2% is achieved by P2R-4B when combining P2R training with P2R inference on V-Star, compared with 84.8% under direct CoT prompting.Replacing direct CoT with P2R inference raises Qwen3-VL-Instruct-4B from 81.7% to 89.0% and P2R-4B from 84.8% to 93.2%.
  • Effect of PRA-GRPO Training: 93.2% for P →R exceeds 90.6% for R →P on V-Star, while alternating updates over both roles outperform optimizing either role alone.Both single-role strategies improve over the no-training baseline, but alternating perception- and reasoning-focused updates produce further gains.
  • Training Dynamics: During the Perceiver phase, V-Star Hit Rate and Avg. IoU improve steadily, alongside high-resolution benchmark average and MME-RealWorld-Lite performance.Hit Rate equals 1 when a predicted-box center falls inside the ground-truth box, serving as a localization proxy.
  • Generalization to Grounding Tasks: P2R-4B improves ReasonSeg Acc@0.5 by 0.5% on the test split and 1.8% on the validation split, for an average gain of 1.1%.P2R uses no grounding-specific data or ground-truth box annotations, relying only on final-answer supervision from fine-grained visual reasoning data.

5 Conclusion

The paper introduces P2R to decouple perception from reasoning for fine-grained visual reasoning, alongside PRA-GRPO, a training strategy aligned with this formulation. Built on Qwen3-VL-Instruct models, P2R performs strongly on challenging high-resolution benchmarks and improves broader multimodal performance.

  • P2R is a unified framework that decouples perception from reasoning for fine-grained visual reasoning.
  • PRA-GRPO is a training strategy aligned with P2R’s decoupled formulation.
  • P2R, built on Qwen3-VL-Instruct models, performs strongly across challenging high-resolution fine-grained visual reasoning benchmarks and improves broader multimodal performance.

Limitations

The work identifies four limitations: increased inference cost, unexplored large-scale PRA-GRPO training, sparse final-answer supervision, and an evaluation limitation stated only partially in the passage.

  • Inference cost: P2R’s two-stage pipeline introduces additional inference cost compared with direct prompting.The paper nevertheless characterizes P2R as simple and effective.
  • Training scale: PRA-GRPO has not been explored at larger training scales, leaving its full scaling behavior unclear.This limitation is attributed to limited computational resources.
  • Supervision: PRA-GRPO uses only final-answer supervision, avoiding bounding box annotations but providing a sparse learning signal.The tradeoff is between annotation requirements and supervision density.
  • Evaluation: The passage introduces a final evaluation limitation but truncates before specifying its content.The supplied text ends at “our evaluation m”.

Ethics Considerations · A Diagnostic Study

The paper highlights privacy risks from stronger fine-grained perception and cautions that P2R’s intermediate visual outputs are not fully faithful explanations. A diagnostic study on V-Star finds that providing ground-truth localization evidence substantially improves Qwen3-VL-4B accuracy, indicating a strong perception bottleneck.

  • Ethics Considerations: Improved fine-grained perception could benefit useful applications while increasing privacy risks in sensitive high-resolution images.Such models may identify small or sensitive visual details.
  • Ethics Considerations: P2R’s bounding boxes and cropped regions should not be interpreted as fully faithful explanations of model decisions.These intermediate outputs accompany the framework but do not guarantee faithful explanations.
  • A Diagnostic Study: Direct CoT uses only the input image, whereas Oracle Hint additionally provides ground-truth bounding boxes and cropped regions.The bounding boxes come from the official V-Star annotation file, with corresponding cropped regions.
  • A Diagnostic Study: The diagnostic study probes whether fine-grained visual reasoning is limited more by reasoning or perception.It compares standard Direct CoT with Oracle Hint using Qwen3-VL-4B on V-Star.
  • A Diagnostic Study: 90.6% overall accuracy is achieved with Oracle Hint, compared with 81.7% for Direct CoT on V-Star using Qwen3-VL-4B.The diagnostic result reports an improvement from 81.7% to 90.6%.
  • A Diagnostic Study: Oracle Hint yields consistent gains on both attribute and spatial questions.The supplied passage states this qualitative pattern alongside the overall accuracy improvement.

B Methodology Details … C Training Details

P2R inference decouples evidence localization from question answering by using Perceiver and Reasoner stages, while PRA-GRPO alternates role-specific updates using final-answer correctness. The procedure highlights predicted regions and crops local patches for reasoning, with training freezing the inactive role and updating only the active role.

  • B.1 P2R Inference Details: P2R inference uses a Perceiver stage to localize question-relevant evidence and a Reasoner stage to answer from that evidence.The two-stage inference process is implemented with separate prompts for evidence localization and reasoning.
  • B.1 P2R Inference Details: The Perceiver identifies question-relevant objects and outputs their 2D bounding boxes and labels in JSON format.The localization prompt asks the model to observe the image first and notes that each object type appears only once.
  • B.1 P2R Inference Details: The Reasoner receives the question together with highlighted and cropped key visual regions and is instructed to think step by step.Predicted boxes are rendered on the original image and converted into local patches before reasoning.
  • B.2 PRA-GRPO Details: PRA-GRPO alternates Perceiver and Reasoner phases, optimizing the active role while freezing the other role from the previous-stage checkpoint.Each phase updates only its active role.
  • B.2 PRA-GRPO Details: For each sample, PRA-GRPO draws grouped rollouts from the active role and computes binary rewards from final-answer correctness.The resulting rewards are converted into group-relative advantages under GRPO.
  • C Training Details: To control visual context, the system renders all predicted boxes on the original image but crops local patches from only the first three boxes.Bounding boxes are parsed from model outputs with regular expressions.
  • B.2 PRA-GRPO Details: During Perceiver rollouts, sampled boxes are passed to the frozen Reasoner, whereas Reasoner rollouts use boxes from the frozen Perceiver.The role-specific rollout inputs differ according to which role is active.
  • C Training Details: The training algorithm computes group-relative advantages as (r_i − µ)/(σ + ϵ) and applies the GRPO objective to update the active role.Rewards are first normalized using the group mean and standard deviation.

C.1 Training Dataset · C.2 Detailed Training Setup · D Evaluation Details

The training data comprise a 10K-example mixture spanning easy to hard fine-grained perception cases, while PRA-GRPO uses shared Perceiver–Reasoner configurations with prompt- and response-length differences. The supplied passages do not provide evaluation details for Section D.

  • C.1 Training Dataset: 10K examples are sampled from DeepEyes, VisualProbe, and ZwZ to provide complementary supervision for fine-grained perception and evidence localization.The mixture contains 3K DeepEyes, 3K VisualProbe, and 4K ZwZ examples.
  • C.1 Training Dataset: DeepEyes contributes 3K relatively easy examples designed to teach basic evidence-localization behavior.Its filtering considers difficulty, answer validity, and perception utility.
  • C.1 Training Dataset: ZwZ contributes 4K medium-difficulty examples generated by distilling teacher-created question-answer pairs from micro-cropped regions back to full images.The samples include explicit region grounding and emphasize subtle local details.
  • C.1 Training Dataset: VisualProbe contributes 3K hard examples featuring high-resolution images, very small targets, and numerous distractors.These examples emphasize sparse, localized evidence identification under clutter.
  • C.1 Training Dataset: The dataset forms a coarse-to-hard difficulty spectrum from DeepEyes through ZwZ to VisualProbe, while using only 10K total training examples.The passages associate this compact mixture with significant performance gains and strong data efficiency.
  • C.2 Detailed Training Setup: PRA-GRPO is implemented with VeRL, and the Perceiver and Reasoner share the same training configuration except for prompt and response length limits.The supplied setup passage identifies VeRL as the implementation framework.
  • C.2 Detailed Training Setup: 2048 × 32 × 32 pixels is the maximum image-resolution budget used during training.This cap is applied as the image pixel budget.

D.1 Evaluation Datasets · D.2 Detailed Evaluation Setup · E Additional Analysis

P2R is evaluated across complementary high-resolution and broader real-world multimodal benchmarks, with reproducible greedy decoding and an increased evaluation resolution. The datasets test fine-grained attributes, spatial relations, and diverse multimodal perception and reasoning scenarios.

  • D.1 Evaluation Datasets: P2R is evaluated on V-Star, HR-Bench-4K/8K, and MME-RealWorld-Lite.These benchmarks span high-resolution fine-grained perception and broader real-world multimodal perception and reasoning.
  • D.1 Evaluation Datasets: V-Star contains 191 high-resolution images averaging 2246 × 1582 pixels and tests attribute recognition and spatial relationship reasoning.Its manually curated questions require accurate visual grounding because the answer cannot be reliably guessed without locating the relevant evidence.
  • D.1 Evaluation Datasets: HR-Bench evaluates Fine-grained Single-instance Perception and Fine-grained Cross-instance Perception with 100 samples per sub-task.Results are reported on HR-Bench-4K using cropped 4K images and HR-Bench-8K using original 8K images.
  • D.1 Evaluation Datasets: MME-RealWorld-Lite uses 50 samples per task, or all samples when a task has fewer than 50 examples.It covers OCR in the wild, remote sensing, diagrams and tables, autonomous driving, and monitoring.
  • D.1 Evaluation Datasets: Together, the benchmarks assess P2R in fine-grained high-resolution settings and general real-world multimodal understanding scenarios.The evaluation suite combines difficult visual grounding with broader perception and reasoning coverage.
  • D.2 Detailed Evaluation Setup: Greedy decoding with temperature 0 is used to ensure reproducible evaluation results.The maximum image resolution is increased to 4096 × 32 × 32 pixels at evaluation time.

E.1 Additional Training Dynamics … E.5 More Cases

Across additional analyses, P2R improves throughout alternating training, depends on accurate localized regions, and maintains an efficiency-accuracy advantage while revealing failure modes in perception and reasoning. GRPO is preferred over DAPO because DAPO collapses Perceiver outputs toward single-box predictions.

  • E.1 Additional Training Dynamics: Accuracy steadily improves on V-Star, HR-Bench-4K, and HR-Bench-8K during both Perceiver and Reasoner phases of PRA-GRPO.The results suggest downstream reasoning improvements can further enhance overall fine-grained visual perception.
  • E.1 Additional Training Dynamics: Bounding-box count and size rise and fall rather than monotonically increasing, indicating exploration followed by more selective localization.These statistics also change during Reasoner training because role interaction influences the Perceiver’s final bounding-box behavior.
  • E.2 Efficiency Analysis: P2R improves benchmark performance over corresponding Qwen3-VL-Instruct base models while retaining relatively high inference efficiency and outperforming the official Thinking with Images tool-use variant.Direct efficiency comparisons with visual search methods are excluded because differing multi-stage pipelines and backends confound wall-clock measurements.
  • E.3 Analysis of Bounding Box Quality: Bounding boxes substantially improve performance over no bounding boxes, whereas random boxes hurt performance, demonstrating that accurate relevant-region localization is critical.For P2R-4B, self-generated boxes reach 93.2% versus 93.7% for oracle boxes, a 0.5% gap.
  • E.4 GRPO vs. DAPO: DAPO rapidly reduces Perceiver bounding-box count toward one, which is unsuitable for tasks requiring multiple boxes, such as comparing two objects’ spatial positions.The paper therefore uses GRPO instead of DAPO in all main experiments, hypothesizing that removing KL regularization causes this collapse.
  • E.5 More Cases: Successful P2R-4B cases span attribute recognition, spatial relations, charts, remote sensing, OCR, and reasoning over small or distant visual evidence.The examples consistently show the model first localizing a compact query-relevant region and then reasoning over it.
  • E.5 More Cases: In one failure, P2R-4B localizes the clock tower correctly but misreads the hour hand as 12 instead of 11 during fine-grained recognition.This demonstrates that accurate localization does not guarantee correct reasoning from the crop.
  • E.5 More Cases: In another failure, partial localization of a river omits spatial context, creating ambiguity between the answers “left” and “upper.”Capturing the full left-side river region would likely avoid this ambiguity.
Loading 2607.01191v1…