PaperlayerView exact revision on arXiv

Source-linked AI summary

MM-Verify: Enhancing Multimodal Reasoning with Chain-of-Thought Verification

Linzhuang Sun, Hao Liang, Jingxuan Wei, Bihui Yu, Tianpeng Li, Fan Yang, Zenan Zhou, Wentao Zhang

arXiv:2502.13383v1cs.CLcs.CVcs.LG

TL;DR

Multimodal reasoning lacks strong verifiers and long-CoT mathematics data. The paper synthesizes data to train MM-Verifier and MM-Reasoner, reporting stronger benchmark performance and improved reasoner performance with more data; their combination surpasses GPT-4o on MathVista.

  • Problem

    Multimodal mathematics lacks robust verifiers and long-CoT data, despite the reported value of self-critique and long-CoT data for text-only reasoning.

    Source: §1 · p. 2

  • Method

    The paper synthesizes long-CoT verification data using tree search and verification, and more efficiently synthesizes multimodal inference data to train MM-Verifier and MM-Reasoner.

    Source: §1 · p. 2

  • Results

    MM-Verifier outperforms larger models on MathCheck, MathVista, and MathVerse; MM-Reasoner improves with dataset size, and their combination surpasses GPT-4o on MathVista.

    Source: §1 · p. 2, §5 · p. 8

  • Takeaways & Limitations

    The reported results support the effectiveness of MM-Verifier and MM-Reasoner for enhancing multimodal reasoning tasks.

    Source: §5 · p. 8

    Scaling tests covered fewer than 100K samples, and limited funding and compute prevented scaling MM-Verifier and MM-Reasoner to Qwen2-VL-72B.

    Source: §6 · p. 9

Abstract

from arXiv · show

According to the Test-Time Scaling, the integration of External Slow-Thinking with the Verify mechanism has been demonstrated to enhance multi-round reasoning in large language models (LLMs). However, in the multimodal (MM) domain, there is still a lack of a strong MM-Verifier. In this paper, we introduce MM-Verifier and MM-Reasoner to enhance multimodal reasoning through longer inference and more robust verification. First, we propose a two-step MM verification data synthesis method, which combines a simulation-based tree search with verification and uses rejection sampling to generate high-quality Chain-of-Thought (COT) data. This data is then used to fine-tune the verification model, MM-Verifier. Additionally, we present a more efficient method for synthesizing MMCOT data, bridging the gap between text-based and multimodal reasoning. The synthesized data is used to fine-tune MM-Reasoner. Our MM-Verifier outperforms all larger models on the MathCheck, MathVista, and MathVerse benchmarks. Moreover, MM-Reasoner demonstrates strong effectiveness and scalability, with performance improving as data size increases. Finally, our approach achieves strong performance when combining MM-Reasoner and MM-Verifier, reaching an accuracy of 65.3 on MathVista, surpassing GPT-4o (63.8) with 12 rollouts.

1 Introduction

The introduction identifies gaps in multimodal verification and long-CoT data, then proposes synthetic-data methods for MM-Verifier and MM-Reasoner. It reports strong verifier benchmarks, improved reasoner performance with more training data, and gains from combining the models.

1 Introduction

  • MM-Verifier achieves state-of-the-art performance on MathCheck, surpasses closed-source models, and beats Qwen2-VL-72B across all metrics on MathVista and MathVerse.Figure 1 reports that the 7B verifier outperforms the other models shown on MathCheck Outcome-Judging.Source: Figure 1 · p. 1, §1 · p. 2
  • The motivation is that multimodal models lack robust verifiers and long-CoT mathematics data, despite reported benefits of self-critique and long-CoT data in text-only settings.Source: §1 · pp. 1–2
  • The paper synthesizes verification data through simulation-based tree search, GPT-4 verification, and rejection sampling, and creates multimodal reasoning data by pairing geometric shapes with textual descriptions.Source: §1 · p. 2
  • MM-Reasoner, trained exclusively on synthetic CoT data, outperforms baseline models and improves as its training dataset grows, though it does not reach state-of-the-art performance.Source: §1 · p. 2
  • The combined 7B MM-Verifier and MM-Reasoner outperform GPT-4 and human performance on MathVista.Source: §1 · p. 2

2 Related Work

Related work covers multimodal language models for mathematics and reward models used to assess reasoning outputs. It distinguishes broader multimodal integration and math-specialized systems from outcome- and process-based evaluation.

2.1 MLLMs for Mathematics

  • MLLM research combines vision encoders with language models, while adapters and instruction-tuned datasets support multimodal understanding and mathematical problem solving.Examples include CLIP, LLaVA, and BLIP; instruction-tuned models include LLaVA-Next, LLaVA-OneVision, Qwen2-VL, and Qwen2.5-VL.Source: §2.1 · pp. 2–3
  • Math-focused MLLMs use symbolic data, multimodal encoders, geometric problem-solving, or math-specific encoders and datasets to improve mathematical capability.Examples include UniMath, G-LLaVA (Gao et al., 2023), MAViS (Zhang et al., 2024e), and EAGLE (Li et al., 2024b).Source: §2.1 · p. 3

2.2 LLM-as-a-Judge

  • Reward models assess outputs to guide optimization or reasoning; ORMs score final answers, whereas PRMs evaluate individual reasoning steps and provide step-level feedback.Source: §2.2 · p. 3
  • Prior PRM work includes automatically supervised step scoring, natural-language feedback, consensus filtering, and multimodal Monte Carlo rollouts.Examples include Math-Shepherd (Wang et al., 2024a), MATHMinos-PRM (Gao et al., 2024), Qwen2.5-Math-PRM (Zhang et al., 2025), and Liu et al. (2024).Source: §2.2 · p. 3

3 Methodology

The methodology constructs MM-Verifier through staged verification-data synthesis and develops MM-Reasoner by using long-CoT data from pure-text reasoning. It combines simulation-based search, verification and filtering with a separate text-to-multimodal data path.

3.1.1 Source Data Collection

  • The question pool draws multimodal math problems from MATH360V categories, while short or answer-only solutions motivate generating varied-length CoT verification trajectories.Source: §3.1.1 · p. 4

3.1.2 Simulation-based Search Algorithm

  • The simulation-based search adapts MCTS because multimodal models may provide unreliable rewards, using repeated simulation correctness ratios as rewards for further search.Source: §3.1.2 · p. 4

3.1.3 Long COT Verification Data Synthesize

  • GPT-4o labels question–solution pairs; answer extraction retains cases where the verifier marks matching answers correct and mismatching answers incorrect for Qwen2-VL-7B-Instruct SFT.Source: §3.1.3 · p. 5

3.2 Stage 2: Rejection Sampling Further Improves Verification

  • Stage 2 uses the Stage 1 verifier to generate additional long-CoT verification examples, string-matches answers for filtering, then trains the verifier further.Source: §3.2 · p. 5

3.3 Bridging the Gap Between Text and MM

  • For MAVIS-GEOMETRY, textualized geometric questions feed Qwen-QwQ; filtered outputs supervise Qwen2-VL-7B-Instruct, avoiding costly tree-search synthesis.Source: §3.3 · p. 6

4 Experiments

The experiments evaluate MM-Verifier and MM-Reasoner across multimodal math benchmarks, examining verifier effectiveness, data scaling, and their combined performance. The combined approach reports higher MathVista accuracy than GPT-4o and human performance, while a case study illustrates step-by-step error detection.

4.1 Experiment Setting

  • Experiments use Qwen2-VL-Instruct-7B as the base model, compare against voting, human, closed-source, and open-source models, and evaluate MathVista, MathVerse, and MathCheck.The settings specify a 4096-token maximum, top-k 5, temperature 0.3, repetition penalty 1.05, and 8 NVIDIA H20 GPUs.Source: §4.1 · pp. 6–7

4.2 Effectiveness of MM-Verifier and MM-Reasoner

  • MM-Verifier outperforms Majority Voting and Qwen2-VL-72B-Instruct on MathVista and MathVerse, and MM-Reasoner outperforms Qwen2-VL-Instruct-7B and LLaMA-3.2-11B-Vision across evaluation metrics.Qwen2-VL-72B-Instruct performance drops when judging MM-Reasoner outputs, while MM-Verifier maintains robust performance on longer outputs.Source: §4.2 · p. 7

4.3 Scalability of Our MM-Reasoner

  • MM-Reasoner performance improves across evaluation settings as training data grows from 6,952 to 32,146 samples, and Verifier-S.2 outperforms Verifier-S.1 in all cases.Both verifier stages improve as training data grows.Source: §4.3 · p. 7

4.4 MM-Verifier and MM-Reasoner achieved SOTA Performance

  • MM-Reasoner generates 12 reasoning rollouts per query, which MM-Verifier evaluates and filters to produce responses.The paper describes this iterative verification-refinement process as improving reasoning precision and robustness.Source: §4.4 · p. 7
  • 65.3 overall accuracy on MATHVISTA surpasses human performance (60.3) and GPT-4o (63.8), while MATHVERSE overall score 25.7 exceeds Math-LLaVA-13B (22.9) and LLaVA-OneVision (20.7).Table 4 compares the combined MM-Verifier and MM-Reasoner with open- and closed-source MLLMs and other baselines.Source: §4.4 · pp. 7–8, Table 4 · p. 8

4.5 Case Study

  • In the case study, MM-Verifier identifies logical errors through step-by-step verification, while Qwen2-VL-72B-Instruct fails to provide a step-by-step reasoning trajectory and misses the mistakes.Figure 5 presents the case study.Source: §4.5 · p. 8, Figure 5 · p. 8

5 Conclusion

The conclusion presents two synthetic-data methods for training MM-Verifier and MM-Reasoner. It reports verifier gains over larger models, scaling with more data, and a combined MathVista result above GPT-4o.

  • Two data-synthesis methods generate long COT verification data and more efficiently synthesized long COT inference data for training MM-Verifier and MM-Reasoner.Source: §5 · p. 8
  • MM-Verifier outperforms larger models on MathCheck, MathVista, and MathVerse, while MM-Reasoner performance improves as data size increases.Source: §5 · p. 8
  • Combining MM-Verifier and MM-Reasoner surpasses GPT-4o on the MathVista benchmark.Source: §5 · p. 8

6 Limitations

The authors report resource-related limits on model size and training-data scale, and plan further experiments when resources become available.

  • Limited funding and computational resources prevented scaling MM-Verifier and MM-Reasoner to Qwen2-VL-72B.Source: §6 · p. 9
  • Scalability tests were restricted to datasets with fewer than 100K samples.Source: §6 · p. 9
  • The authors plan further experiments when additional computational resources become available.Source: §6 · p. 9

A Performance of Naive MCTS in Multimodal Reasoning

The appendix reports that simple MCTS did not improve multimodal reasoning performance on MathVista. The authors attribute this to greater hallucination prevalence in multimodal models and describe a simulation-based tree-search design.

B Detailed Performance of MM-Verifier on Sub-tasks of MathVista

MM-Reasoner with MM-Verifier (Stage 2) achieves higher overall MathVista accuracy than the compared models across the reported sample settings. Verification improves performance over Majority Voting across models, with MM-Reasoner benefiting most.

B Detailed Performance of MM-Verifier on Sub-tasks of MathVista

  • The comparison covers GPS, MWP, FQA, TQA, and VQA sub-tasks on MathVista.The evaluated models are Qwen2-VL-Instruct-7B, LLaMA-3.2-11B-Vision, and MM-Reasoner.Source: B Detailed Performance of MM-Verifier… · p. 12
  • 61.5 overall accuracy for MM-Reasoner with MM-Verifier (Stage 2) in Sample 4 surpassed Qwen2-VL’s 59.8 and LLaMA-3.2-11B-Vision’s 50.4.MM-Reasoner obtained 65.3 and 65.7 in Sample 8 and Sample 12, respectively.Source: B Detailed Performance of MM-Verifier… · p. 12
  • Across all models, MM-Verifier (Stage 2) consistently improved performance over Majority Voting, with MM-Reasoner benefiting the most.The passage attributes the gains to verification and says MM-Reasoner effectively integrates verification feedback into its reasoning pipeline.Source: B Detailed Performance of MM-Verifier… · p. 12
  • In a presented case, MM-Verifier correctly verified an answer using CoT while Qwen2-VL-72B-Instruct failed to.The paper describes the solution as providing only a simple answer.Source: Figure 6 · p. 12

C Prompts

The section identifies three prompts: one for generating Verify data, one for distilling QwQ-32B-Preview after converting multimodal data to text, and one for extracting answer data.

C Prompts

  • The three prompts cover Verify-data generation, QwQ-32B-Preview distillation after multimodal-to-text conversion, and answer extraction.They are shown in Figures 7, 8, and 9, respectively.Source: C Prompts · p. 12

D More Case Study

A case study contrasts MM-Verifier’s CoT-based verification with Qwen2-VL-72B-Instruct’s classification of a solution. The section also identifies the scope of two MathVista tables and three prompts.

D More Case Study

  • MM-Verifier correctly classified a solution with only a simple answer by leveraging CoT, whereas Qwen2-VL-72B-Instruct incorrectly classified it as correct.Figure 6 presents this case.Source: D More Case Study · p. 12
  • Table 5 compares Qwen2-VL-Instruct-7B, LLaMA-3.2-11B-Vision, and MM-Reasoner across GPS, MWP, FQA, TQA, and VQA on MathVista.The sub-task labels denote geometry problem solving, math word problem, figure question answering, textbook question answering, and visual question answering.Source: Table 5 · p. 13
  • Table 6 compares MCTS with Majority Voting against MM-Verify on MathVista.Source: Table 6 · p. 14
  • Figures 7–9 show the Verify Label, QwQ-32B-Preview Distill, and Answer Extract prompts, respectively.Source: Figure 7 · p. 14, Figure 8 · p. 14, Figure 9 · p. 15
Loading 2502.13383v1…