Source-linked AI summary
MM-Eureka: Exploring the Frontiers of Multimodal Reasoning with Rule-based Reinforcement Learning
Fanqing Meng, Lingxiao Du, Zongkai Liu, Zhixiang Zhou, Quanfeng Lu, Daocheng Fu, Tiancheng Han, Botian Shi, Wenhai Wang, Junjun He, Kaipeng Zhang, Ping Luo, Yu Qiao, Qiaosheng Zhang, Wenqi Shao
TL;DR
Multimodal reasoning research lacks stable, large-scale rule-based RL and high-quality, diverse mathematical data. The paper introduces MMK12 and MM-Eureka, using verified multimodal problems, online filtering, and two-stage training; MM-Eureka achieves strong multidisciplinary and mathematical reasoning results while remaining behind closed-source models on some challenging benchmarks.
Problem
Prior multimodal RL efforts use limited tasks or modest scales, while high-quality, diverse multimodal mathematical reasoning datasets remain scarce.
Method
The paper constructs MMK12 with verified answers and solution processes, then trains MM-Eureka using rule-based RL with online filtering and, for 32B, a two-stage strategy.
Results
MM-Eureka-32B achieves strong multidisciplinary and mathematical reasoning performance, ranking second to o1 on multidisciplinary K12 evaluation and outperforming most comparable models.
Takeaways & Limitations
The results support stable rule-based RL as an effective approach for open-source multimodal reasoning across mathematical and multidisciplinary tasks.
Takeaways & Limitations
MM-Eureka-32B still has considerable gaps behind closed-source models on challenging benchmarks such as MathVision and OlympiadBench.
Abstract
from arXiv · showhide
DeepSeek R1, and o1 have demonstrated powerful reasoning capabilities in the text domain through stable large-scale reinforcement learning. To enable broader applications, some works have attempted to transfer these capabilities to multimodal reasoning. However, these efforts have been limited by the limited difficulty of selected tasks and relatively small training scales, making it challenging to demonstrate strong multimodal reasoning abilities. To address this gap, we introduce the MMK12 dataset and MM-EUREKA with 7B and 32B parameters. The former is a high-quality multimodal mathematics reasoning dataset featuring diverse knowledge domains with human-verified answers and solution processes. The latter is a multimodal model employing rule-based reinforcement learning on MMK12, utilizing online filtering and two-stage training strategy to enhance training stability. MM-EUREKA demonstrates remarkable performance gains in multimodal mathematical reasoning, outperforming previous powerful models like InternVL2.5-78B or InternVL2.5-38B-MPO. In particular, MM-EUREKA achieves competitive or superior performance compared to both open-source and closed-source models, and trails slightly behind o1 in multidisciplinary reasoning tasks. We open-source our complete pipeline to foster further research in this area. We release all our codes, models, data, etc. at https://github.com/ModalMinds/MM-EUREKA
1 INTRODUCTION
The paper addresses the unresolved challenge of stable, large-scale rule-based reinforcement learning for multimodal reasoning. It introduces MMK12 and MM-Eureka, combining verified multimodal mathematics data with online filtering and staged training to achieve strong multidisciplinary performance.
- Motivation: Stable large-scale rule-based RL remains an important unsolved problem for open-source multimodal reasoning models.Prior efforts used modest model sizes, limited tasks, or lacked verification in large-scale image-text training.
- Results: MM-Eureka-32B scores 72.3 on multidisciplinary K12 evaluation, ranking second to o1’s 73.9 and exceeding Qwen2.5-VL-72B’s 70.3.MM-Eureka-7B scores 66.1 on MMK12 evaluation and 73.0 on MathVista, surpassing InternVL2.5-VL-78B on MathVista.
- Training: Online filtering removes zero-advantage prompts, while MM-Eureka-32B uses two-stage RL with MMK12 followed by Geo3k and KL regularization.The strategy is designed to reduce collapse, improve stability, and address domain-specific deficiencies.
- Findings: The authors report that mathematical RL training improves performance across physics, chemistry, biology, and other disciplines, and generalizes better than SFT and COT SFT.They attribute improvement primarily to increased probability of generating correct answers rather than acquiring new knowledge.
- Resources: The paper open-sources its models, code, and multimodal mathematical reasoning data, supporting further community exploration.The repository includes broader RL algorithm support and higher-quality data than existing open-source repositories.
2 RELATED WORK
Related work establishes large-scale RL as a route to stronger text reasoning, while multimodal reasoning remains an emerging area with leading systems largely closed-source. Recent multimodal RL efforts have begun exploring visual reasoning but remain limited in scale and scope.
- Text-domain reasoning: Large-scale RL methods such as PPO, GRPO, RLOO, and Reinforce++ have been developed to improve language-model reasoning.DeepSeek R1 showed that pure RL can encourage behaviors including self-reflection and error correction.
- Multimodal reasoning: Closed-source systems including GPT-4o and Kimi-VL lead multimodal reasoning, while open-source research remains comparatively early.Concurrent work seeks to enhance visual reasoning and trigger an “Aha Moment” in vision-language models.
3 MMK12: MULTIMODAL MATHEMATIC K12-LEVEL DATASET
MMK12 is designed to address limited scope and answer-reliability problems in multimodal mathematics datasets. It combines diverse K12 problems, standardized construction, verified solutions, and multidisciplinary evaluation.
- Dataset motivation: Existing datasets trade off diversity, scope, and answer reliability: some focus only on geometry, others use synthetic data, and some lack guaranteed reference correctness.These limitations constrain multimodal mathematical reasoning evaluation and training.
- Dataset scope: MMK12 contains over 15,000 multimodal mathematical reasoning problems across geometry, functions, and graphical reasoning.Each problem includes a standard reference answer and detailed step-by-step solution.
- Construction: The construction process collects problems from Chinese textbooks and examinations, translates and refines them into English with LLM assistance, and verifies answers with Math-Verify.Only fill-in-the-blank problems are retained for training to minimize false positives.
- Dataset composition: The training set has 15,616 multimodal fill-in-the-blank mathematics problems, while the evaluation set has 2,000 multiple-choice questions across four disciplines.Training samples include questions, images, final answers, and CoT-formatted solution processes.
- Quality and evaluation: Human involvement and verification support both question diversity and correctness of answers and solution processes.The multidisciplinary evaluation set is intended to provide convenient and accurate testing of reasoning capabilities.
4 METHOD
MM-Eureka combines rule-based rewards and GRPO with online filtering to stabilize multimodal RL, while larger models additionally use two-stage training with KL regularization and domain-specific data.
- 4.3 ADVANTAGE ESTIMATION AND POLICY UPDATE: Rule-based RL uses accuracy and format rewards with GRPO, which optimizes policies from relative performance among grouped responses without a critic model.Accuracy is binary, while format rewards are 0 or 0.5; GRPO uses intra-group reward comparisons.
- 4.3 ADVANTAGE ESTIMATION AND POLICY UPDATE: The policy update further clips ratios to [0, c] with c=3 to mitigate loss spikes caused by large policy ratios combined with negative advantages.This constraint is applied before the PPO-clip loss update.
- 4.4 ONLINE FILTERING: Online filtering removes prompts whose responses are completely correct or incorrect, because their GRPO advantages are zero and provide insufficient gradient information.The filter dynamically retains prompts with nonzero learning signal for subsequent RL updates.
- 4.4 ONLINE FILTERING: Online filtering stabilizes training: filtered models maintain increasing accuracy rewards and response lengths, whereas unfiltered models eventually approach zero accuracy and shorten responses.The ablation associates filtering with prevention of model collapse during optimization.
- 4.5 TWO-STAGE TRAINING: For MM-Eureka-32B, training becomes harder as scale increases, with sudden near-zero reward collapses preceded by sharp policy-ratio increases.The 32B model trained only on MMK12 also shows performance degradation compared with the 7B model’s broader improvement.
- 4.5 TWO-STAGE TRAINING: The two-stage strategy first applies GRPO without KL regularization on MMK12, then adds KL regularization and Geo3k data to constrain updates and address domain-specific deficiencies.The first stage targets general reasoning, while the second stage improves stability and supplements domain coverage.
5 EXPERIMENTS
MM-Eureka is evaluated across multimodal mathematical benchmarks and multidisciplinary MMK12 tasks against closed-source and open-source baselines. Both model sizes perform strongly among similarly sized open-source systems, while MM-Eureka-32B retains gaps on the most challenging comparisons with closed-source reasoning models.
- Evaluation setup: Evaluations cover MathVista, MathVerse, MathVision, OlympiadBench, WeMath, and the multidisciplinary MMK12 benchmark.MMK12 additionally assesses fundamental multimodal problems across mathematics, physics, chemistry, and biology.
- Mathematical benchmarks: 73.0 on MathVista places MM-Eureka-7B above InternVL-78B by 0.7% and InternVL2.5-38B-MPO by 0.2%.The reported comparison concerns multimodal mathematical reasoning performance.
- Mathematical benchmarks: MM-Eureka-32B exceeds comparable-size open-source models on every benchmark except MathVision and approaches Qwen-72B-Instruct’s average performance.It also outperforms Claude3.7 Sonnet on WeMath.
- Closed-source comparison: MM-Eureka-32B remains behind closed-source reasoning models on harder benchmarks such as MathVision and OlympiadBench, despite outperforming o1 on MathVista.The paper identifies considerable performance gaps against closed-source models overall.
- Multidisciplinary evaluation: MM-Eureka-32B is 1.7% behind o1 on multidisciplinary MMK12 evaluation while outperforming larger models including Qwen-2.5-VL-72B and Gemini2-Flash-Thinking.MM-Eureka-7B also exceeds InternVL2.5-VL-78B overall and is only slightly behind Qwen-2.5-VL-32B.
- Qualitative results: Qualitative examples show MM-Eureka-32B applying known concepts and performing multi-step deduction more effectively than Qwen2.5-VL-32B-Instruct across four subjects.The comparison includes mathematics, physics, chemistry, and biology.
6 DISCUSSION
The discussion examines how reinforcement learning changes reasoning behavior, generalizes beyond mathematics, and can be stabilized through staged training. It attributes gains primarily to better use of existing knowledge while identifying domain-specific regressions and collapse risks during training.
- Knowledge and reasoning: RL improvements appear to increase the probability of producing correct answers rather than acquiring new knowledge.Figure 4 reports larger gains for problems initially answered correctly at least once, while zero-correct problems remain nearly unchanged.
- Post-training comparison: RL generalizes better than SFT and COT SFT, especially on out-of-distribution physics, chemistry, and biology test sets.The comparison uses identical data and consistent RL settings for the post-training strategies.
- Training strategy: The two-stage strategy uses K12 RL for general reasoning, then Geo3k training to address specific deficits while adding KL divergence for stability.Stage 1 omits KL divergence to accelerate training; Stage 2 adds it to maintain stability.
- Training stability: Continuous RL without KL divergence can cause sudden collapse through policy-ratio instability, whereas second-stage KL regularization supports steadily increasing accuracy rewards.The reported mechanism is excessive updating in the new policy, reflected by sharp ratio fluctuations before collapse.
- Training strategy: After the first training stage, MathVista geometry performance drops from 74.0 to 56.7 before Geo3k second-stage training restores broader benchmark performance.The regression motivates the domain-targeted second stage.
7 CONCLUSION
MM-Eureka develops 7B and 32B multimodal reasoning models that achieve top performance among similarly sized models, with the 32B model nearly matching o1 on multidisciplinary tasks.
- MM-Eureka-7B and MM-Eureka-32B are presented as top performers among similarly sized multimodal reasoning models.
- MM-Eureka-32B surpasses almost all evaluated open-source and closed-source models on multidisciplinary reasoning tasks, trailing only slightly behind o1.
A APPENDIX: QUALITATIVE ANALYSIS
The appendix presents qualitative examples spanning mathematics, physics, chemistry, and biology, contrasting errors from Qwen2.5-VL-32B-Instruct with correct MM-Eureka-32B reasoning. It also includes representative multimodal questions and solution analyses.
- Mathematics: The appendix includes a spatial-geometry question involving a square column and a perpendicularity condition.
- Qualitative comparisons: Figures 6–9 contrast Qwen2.5-VL-32B-Instruct’s reasoning errors with MM-Eureka-32B’s correct reasoning across mathematics, physics, chemistry, and biology.
- Physics: The qualitative material also presents a spring-force physics question involving 6 kg and 4 kg objects.
- Biology: It includes a biology question about hybridizing a gene strand with messenger RNA and identifying an unreasonable explanation.
- Chemistry: It includes a chemistry question asking which statement is incorrect during titration of oxalic acid with sodium hydroxide.
MM-EUREKA
The appendix analyzes oxalic-acid titration at characteristic points on the pH curve, tracking neutralization stages, solution composition, and acidity or basicity.
- Titration setup: The analysis treats H2C2O4 as a weak dicarboxylic acid titrated by strong-base NaOH through successive neutralization steps.
- Curve interpretation: At 25 mL NaOH, the first equivalence point converts H2C2O4 to NaHC2O4, with pH around 2.
- Curve interpretation: At 40 mL NaOH, the solution lies between equivalence points and contains a mixture of NaHC2O4 and Na2C2O4 at pH 5.5.
- Curve interpretation: At 50 mL NaOH, the second equivalence point converts H2C2O4 to Na2C2O4 at pH 8.7.