PaperlayerView exact revision on arXiv

Source-linked AI summary

GeoPQA: Bridging the Visual Perception Gap in MLLMs for Geometric Reasoning

Guizhen Chen, Weiwen Xu, Hao Zhang, Hou Pong Chan, Deli Zhao, Anh Tuan Luu, Yu Rong

arXiv:2509.17437v1cs.CL

TL;DR

The paper addresses limited MLLM reasoning gains in vision-intensive geometric tasks alongside shortcomings in visual understanding. It assesses basic geometric perception and trains perception before reasoning, reporting gains over direct reasoning training.

  • Problem

    MLLMs often fail to perceive basic geometric information, restricting attainable reward signals and the scope of reasoning improvement in RL.

    Source: §1 · p. 1

  • Method

    A two-stage RL framework first enhances visual understanding of geometric concepts and relationships, then trains the model for reasoning.

    Source: §1 · pp. 1–2

  • Results

    9.7% and 9.1% accuracy improvements in geometric reasoning and problem solving, respectively, over direct single-stage reasoning training on MathVista.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The findings highlight strong perceptual foundations as prerequisites for effective higher-level reasoning in multimodal systems.

    Source: §4 · p. 5

    Stage 1 accuracy evaluation relies on GPT-4o-mini to judge free-form answers, adding API cost and time.

    Source: Limitations · p. 6

Abstract

from arXiv · show

Recent advancements in reinforcement learning (RL) have enhanced the reasoning abilities of large language models (LLMs), yet the impact on multimodal LLMs (MLLMs) is limited. Particularly in vision-intensive tasks like geometric reasoning, MLLMs hallucinate frequently, leading to inaccurate reasoning. We attribute this to the perceptual bottleneck in MLLMs, which caps the benefits of reasoning training. To quantify this, we design a Geo-Perception Question-Answering (GeoPQA) benchmark, targeting basic geometric concepts and spatial relationships. Experiments on GeoPQA reveal significant shortcomings of MLLMs in visual perception, which constrain RL reward signals for effective training. To address this bottleneck, we propose a two-stage RL training framework by first enhancing the visual perception of geometric structures, then fostering reasoning capabilities. Applied to Qwen2.5-VL-3B-Instruct, our two-stage training improves geometric reasoning by 9.7% and geometric problem solving by 9.1%, compared to the direct reasoning training approach. Our method also generalizes to other vision-intensive domains like figure understanding, highlighting the importance of perceptual grounding in effective MLLM reasoning.

1 Introduction

The introduction argues that visual-perception errors limit multimodal models’ performance on vision-intensive reasoning tasks and constrain the benefits of reasoning-focused reinforcement learning. It introduces geometric perception assessment and a perception-first, two-stage training approach, reporting gains on geometric tasks and generalization to other visual domains.

1 Introduction

  • Performance gains from applying reinforcement learning to MLLMs remain limited on vision-intensive reasoning benchmarks, where foundational visual-understanding errors can disrupt later deductions.The introduction illustrates this with errors in identifying rotation angles and composing angles.Source: §1 · p. 1, Figure 1 · p. 1
  • The authors hypothesize that weak geometric perception limits attainable RL reward signals and measure the gap with questions on basic geometric concepts and relationships.Their empirical results show frequent MLLM failures in contrast with near-perfect human performance.Source: §1 · p. 1
  • 9.7% and 9.1% accuracy improvements in geometric reasoning and problem solving, respectively, are reported for two-stage training over direct reasoning training.The approach also outperforms larger open-source MLLMs and prior mathematical visual specialists.Source: §1 · p. 2
  • The proposed framework trains geometric perception before reasoning, using a curated QA dataset based on real and synthetic diagrams.The authors also report generalization to figure and textbook understanding.Source: §1 · p. 2

2 Methodology

The methodology assesses geometric perception and then trains MLLMs in two stages: first on perception-focused questions, then on geometric reasoning. It describes the assessment, training data and reward, and evaluation setup.

2.1 Preliminary Analysis

  • Existing geometric-perception evaluations use limited templates, while this assessment probes shapes, lengths, angles, and spatial relations with easily verified answers.Questions are generated from Geo170K image-caption pairs, and the reported evaluation finds substantial MLLM deficiencies—even for GPT-4o—compared with humans’ accuracy above 90%.Source: §2.1 · p. 2

2.2 Framework Overview

  • The framework first improves geometric perception, then uses that perceptual foundation to train more complex, multistep reasoning.The methodology describes these as perception and reasoning stages.Source: §2.2 · p. 3

2.3 Stage 1: Perception-oriented Training

  • Stage 1 trains on GeoPQA questions for real and synthetic figures, with an all-questions-correct reward judged from the model’s response.GPT-4o filters questions and answers; human inspection found 92% of 100 sampled items valid and high-quality. Multiple questions can be paired per image, and instances with all “yes” answers are downsampled to mitigate reward hacking.Source: §2.3 · p. 3

2.4 Stage 2: Reasoning-oriented Training

  • Stage 2 trains the model on geometric reasoning questions using the Geo170K QA-tuning subset and the standard GRPO setup.It follows the initial perception training stage.Source: §2.4 · p. 3

2.5 Implementation

  • The experiments use Qwen-2.5-VL-3B-Instruct and Qwen-2.5-VL-7B-Instruct, evaluating geometric reasoning and problem solving on MathVista.The comparison includes proprietary and open-source MLLM baselines.Source: §2.5 · p. 3

3 Results and Analyses

The two-stage approach significantly outperforms reasoning-only training by a 9.7% margin.

  • The two-stage approach significantly outperforms reasoning-only training by a 9.7% margin.Source: §3 · p. 3

Model GR GPS

Two-stage perception-then-reasoning training improves geometric reasoning and problem solving over reasoning-only training, with gains also reported at 7B scale and on some visually grounded tasks. Results further compare staged training with mixed training and examine multi-question perception training and transfer across task types.

Main results

  • The reported geometry problem-solving (GPS) improvement is 9.1%; reasoning-only training scores even slightly lower than the original baseline, and perception training bridges the perception gap.Source: Model GR GPS · p. 4
  • The Qwen2.5-VL-3B-Instruct two-stage model establishes new open-source state-of-the-art geometric-task performance and narrows the gap to GPT-4o to around 2%.GPT-4o scores 74.1% on GR and 75.0% on GPS.Source: Model GR GPS · p. 4

Perception and reasoning performance

  • On GeoPQA, reasoning training degrades performance by 15.1%, while the two-stage approach reaches 83.2% perception performance, 15% above baseline.Source: Model GR GPS · p. 4

Impact of training strategy on tasks of different vision intensity

  • Perception-involved methods generally outperform reasoning-only training and the base model across five vision-intensity categories, while staged training beats mixed training in Vision Only.Source: Model GR GPS · pp. 4–5

Impact of multiple perception QAs per image

  • Training with multiple perception questions per image improves downstream GR by 9.6% and GPS by 10.6%, despite slightly lower perception-task performance under its stricter reward.The reward is positive only when all sub-questions for an image are answered correctly.Source: Model GR GPS · p. 5, Table 4 · p. 5

Results on a larger scale

Generalization to other tasks

  • Gains extend to figure QA (+1.5%), textbook QA (+2.6%), and scientific reasoning (+2.5%), but not to numerical commonsense (-2.8%) or math word problems (-1.1%).Source: Model GR GPS · p. 5

4 Conclusion

The conclusion identifies visual perception as a constraint on reinforcement-learning gains in MLLM reasoning. It presents perception-first training as a route to improving geometric reasoning and points to broader directions for future work.

4 Conclusion

  • MLLMs’ visual perception significantly constrains reinforcement learning’s effectiveness for reasoning, a challenge the paper targets with geometric-perception assessment and perception-first training.The conclusion describes strong perceptual foundations as prerequisites for effective higher-level reasoning.Source: §4 · p. 5
  • The paper reports substantial improvements on challenging benchmarks after explicitly enhancing visual perception before reasoning training.The conclusion does not specify the improvement values.Source: §4 · p. 5
  • Future directions include testing the approach with thinking-with-images methods and extending it to vision-intensive tasks such as chart understanding.The conclusion cites OpenAI (2025), Su et al. (2025), and Ma et al. (2024) for thinking-with-images approaches, and Huang et al. (2024) for chart understanding.Source: §4 · p. 5

Limitations

The appendix notes the API cost and time associated with using an LLM judge during stage-one training. It also describes the perception-question prompt and the basic and composite shapes and annotations used in synthetic diagrams.

Limitations

  • Stage-one training uses GPT-4o-mini to judge free-form perception-QA answers, adding API cost and time.The judge parses answers and determines correctness.Source: Limitations · p. 6

A.1 Prompt to generate perception QAs

  • The perception-QA prompt requests image-grounded questions with yes/no, numeric, or simple-string answers, no reasoning, and at most seven questions.It also instructs the generator to avoid duplicate questions and return an empty array when no meaningful question can be generated or the image is too simple.Source: §A.1 · p. 8

A.2 Synthetic geometric diagram generation

  • Synthetic diagrams combine basic shapes, composites of two to four shapes with predefined spatial relationships, and geometric annotations.The listed basics include line segments, circles, triangles, quadrilaterals, and pentagons; annotations include right-angle symbols and equal-side or equal-angle markings.Source: §A.2 · p. 8

A.3 Dataset statistics

The appendix reports GeoPQA’s train/test split and describes how its question-count and question-type distributions characterize dataset complexity and content.

A.3 Dataset statistics

  • GeoPQA contains 5,420 training samples and 659 test samples, with the training set balanced between real-world and synthetic images.Table 6 is identified as showing the real-versus-synthetic distribution.Source: §A.3 · p. 8, Table 6 · p. 8
  • The number of perception questions per image serves as a proxy for visual complexity, with its distribution reported in Table 7.The passage provides no distribution values.Source: §A.3 · p. 8, Table 7 · p. 8
  • GeoPQA questions cover basic geometric elements and relationships, with the question-type distribution reported in Table 8.Elements include shapes, lengths, and angles; relationships include intersection, parallelism, perpendicularity, and tangency.Source: §A.3 · p. 8, Table 8 · p. 8

A.4 Quality control of generated perception QAs

The authors filter perception questions using GPT-4o and inspect a random sample to assess the resulting dataset quality.

A.4 Quality control of generated perception QAs

  • GPT-4o filtering rejects questions when the image lacks the required information or the ground-truth answer contradicts the image description.The evaluation prompt requests a concise rationale and a binary correctness judgment.Source: §A.4 · p. 9
  • 92% of 100 randomly inspected samples were valid and high-quality after GPT-4o filtering.The remaining 8% invalid samples comprised 2% from the synthetic subset and 6% from the real-world image subset.Source: §A.4 · p. 9

B Training setup

The training uses the same hyperparameter settings across all experiments to support fair, direct comparison.

B Training setup

  • The same hyperparameter settings are used across all experiments to ensure fair, direct comparison.Table 9 is identified as the source of the training hyperparameter configuration.Source: B Training setup · p. 9

C Evaluation

Evaluation uses VLMEvalKit for standardized, reproducible metrics; Table 9 is identified as listing training hyperparameters.

C Evaluation

  • All evaluation uses VLMEvalKit to ensure standardized and reproducible evaluation metrics.Source: C Evaluation · p. 9
  • Table 9 lists the hyperparameters used in training.Source: Table 9 · p. 9

D Performance on other tasks

Table 10 compares the proposed method with reasoning-only training on other MathVista tasks.

D Performance on other tasks

Related papers

Loading 2509.17437v1…