Source-linked AI summary
OpenMMReasoner: Pushing the Frontiers for Multimodal Reasoning with an Open and General Recipe
Kaichen Zhang, Keming Wu, Zuhao Yang, Bo Li, Kairui Hu, Bin Wang, Ziwei Liu, Xingxuan Li, Lidong Bing
TL;DR
Multimodal reasoning research lacks transparent and reproducible data-curation and training strategies. OpenMMReasoner introduces an open two-stage SFT-and-RL recipe with curated datasets and systematic design analysis, and reports stronger reasoning performance while identifying data quality and training design as important factors. Its evidence is primarily limited to the Qwen2.5-VL-Instruct family and image-domain evaluation.
Problem
Multimodal reasoning research lacks transparent, reproducible training pipelines and detailed evidence about data curation and training design.
Method
OpenMMReasoner combines a high-quality SFT dataset with a curated RL dataset and systematic analyses of data curation, sampling, and RL design.
Results
OpenMMReasoner consistently outperforms strong multimodal reasoning methods across a wide range of benchmarks.
Takeaways & Limitations
The study highlights data diversity, structured sampling, difficulty-aware curricula, and well-defined rewards as practical factors associated with more effective multimodal reasoning training.
Takeaways & Limitations
Evaluation primarily uses the Qwen2.5-VL-Instruct family and image-domain tasks, leaving broader modality and scaling generality unresolved.
Abstract
from arXiv · showhide
Recent advancements in large reasoning models have fueled growing interest in extending such capabilities to multimodal domains. However, despite notable progress in visual reasoning, the lack of transparent and reproducible data curation and training strategies remains a major barrier to scalable research. In this work, we introduce OpenMMReasoner, a fully transparent two-stage recipe for multimodal reasoning spanning supervised fine-tuning (SFT) and reinforcement learning (RL). In the SFT stage, we construct an 874K-sample cold-start dataset with rigorous step-by-step validation, providing a strong foundation for reasoning capabilities. The subsequent RL stage leverages a 74K-sample dataset across diverse domains to further sharpen and stabilize these abilities, resulting in a more robust and efficient learning process. Extensive evaluations demonstrate that our training recipe not only surpasses strong baselines but also highlights the critical role of data quality and training design in shaping multimodal reasoning performance. Notably, our method achieves a 11.6% improvement over the Qwen2.5-VL-7B-Instruct baseline across nine multimodal reasoning benchmarks, establishing a solid empirical foundation for future large-scale multimodal reasoning research. We open-sourced all our codes, pipeline, and data at https://github.com/EvolvingLMMs-Lab/OpenMMReasoner.
1. Introduction
OpenMMReasoner addresses limited transparency and reproducibility in multimodal reasoning by developing an open, scalable recipe spanning SFT and RL. Its data curation and training analyses emphasize quality, diversity, and reproducible design.
- 1. Introduction: The work targets a reproducibility gap because prior studies often omit data-curation details and comprehensive ablation analyses.This gap obscures how reasoning-capable multimodal models are built and how their training dynamics evolve.
- 1. Introduction: OpenMMReasoner provides a fully transparent recipe covering both SFT and RL for training multimodal reasoning models.The workflow is designed to be scalable, reproducible, and applicable across reasoning settings.
- 1. Introduction: An 874k-sample SFT dataset with step-by-step validation establishes a strong reasoning foundation before RL.The dataset combines carefully selected teacher-model traces with cross-domain data sources.
- 1. Introduction: A 74k-sample RL dataset and analysis of diverse optimization designs further sharpen and stabilize multimodal reasoning capabilities.The RL stage is presented as a complementary enhancement to the SFT foundation.
- 1. Introduction: The study identifies data diversity, including diversity in answer traces, as a critical factor in curating high-quality multimodal reasoning datasets.It treats source diversity and answer diversity as distinct axes for improvement.
2. Related Work
Prior multimodal reasoning work has advanced RL and emphasized high-quality supervision, but often lacks transparent, reproducible training pipelines. OpenMMReasoner addresses this gap with an open recipe spanning SFT and RL, supported by explicit data and teacher-model pipelines.
- Prior work: Recent multimodal studies extend reinforcement-learning-based reasoning, while supervised fine-tuning work highlights the importance of high-quality supervision.The related work connects multimodal reasoning progress to developments in both RL and SFT.
- Training pipeline: The proposed pipeline collects diverse data, selects teacher models to generate answer traces, and explores RL algorithms and filtering strategies before finalizing the recipe.Figure 2 presents the SFT and RL phases as coordinated training recipes, with optimization choices concentrated in RL.
- Open science: OpenMMReasoner fully open-sources its data pipeline, SFT/RL datasets, and model weights to support transparent reproduction.The comparison concerns the extent of open-sourcing across existing large multimodal reasoning models.
- Teacher supervision: Teacher models significantly improve reasoning ability and data efficiency even when training data are limited.This finding motivates teacher-guided trace generation within the broader data pipeline.
- OpenMMReasoner: OpenMMReasoner addresses limited reproducibility by providing a fully open recipe covering both supervised fine-tuning and reinforcement learning.The paper positions transparent data curation and training design as guidance for building reliable multimodal reasoning models.
3. Supervised Fine-tuning Recipe
The supervised fine-tuning recipe progresses from raw questions through verified reasoning traces and domain mixing to build a strong, efficient reasoning foundation.
- Data curation: Over-filtering answers reduces dataset diversity without improving overall performance.Both difficulty-based and length-based filtering reduce performance, so the recipe adopts no filtering.
- Data distillation: Teacher selection improves supervision quality, with all teacher-based variants outperforming the baseline.Candidate traces are retained only after format validation and answer verification, producing approximately 59k verified traces.
- Data scaling: 50.5 to 55.2 average benchmark performance follows increasing verified answers per question from ×1 to ×8.The final ×8 configuration balances performance gains with manageable data-generation cost.
- Domain mixing: Mixing image- and text-based mathematical supervision improves reasoning performance across multimodal and reasoning benchmarks.The study attributes this to effective reasoning transfer across domains.
- Dataset construction: 874k mixed SFT samples combine distilled general reasoning with image- and text-based mathematical supervision.The dataset grows from 103k raw questions to 583k verified general-reasoning samples before final domain mixing.
- Overall evaluation: The final recipe shows superior performance and data efficiency across nine visual reasoning benchmarks compared with other SFT approaches.Table 6 reports evaluation results for visual reasoning benchmarks and identifies best and second-best open-source results.
4. Reinforcement Learning Recipe
The RL recipe combines curated multimodal data, reward design, and algorithmic choices to improve reasoning stability, efficiency, and benchmark performance. GSPO is selected as the strongest overall algorithm, while RL further improves and transfers reasoning capabilities.
- Dataset Construction: The RL stage uses approximately 74k cleaned samples sourced across science, mathematics, charts, and puzzles, with answer verification and image-text deduplication.The top-performing SFT checkpoint initializes RL training.
- RL Algorithm Selection: GSPO converges faster, earns higher rewards, and behaves more stably than DAPO and GRPO under unified multimodal RL settings.The comparison evaluates stability, exploration, and efficiency.
- Reward Function: The composite reward combines task accuracy and output-format consistency, with λfmt = 0.1 controlling the trade-off in the final configuration.Racc measures correctness, while Rfmt measures format consistency.
- Overall Results: RL improves performance over SFT and reaches state-of-the-art results on WeMath, MathVerse, and MathVista.The reported gains are described as more consistent after RL.
- Analysis and Insights: RL training also improves textual reasoning and balances accuracy with efficiency by avoiding excessively long reasoning trajectories.The model outperforms OVR in reasoning efficiency on MMMU and We-Math.
5. Conclusion
OpenMMReasoner presents a transparent, scalable SFT-and-RL framework for multimodal reasoning. Its findings emphasize data diversity, effective RL design, and reproducible pipelines as central to stronger and more efficient reasoning systems.
- Conclusion: OpenMMReasoner unifies transparent SFT and RL stages while systematically studying data curation, sampling, and RL design for LMRMs.The framework is presented as an open-source empirical reference for scalable multimodal reasoning research.
- Conclusion: Scaling diversity across domains and reasoning traces is more valuable than merely increasing dataset size.The conclusion highlights diversity in data and reasoning traces as a key design principle.
- Conclusion: GSPO, structured sampling, difficulty-aware curricula, and well-defined rewards improve stability, efficiency, precision, and multimodal consistency.These design choices are identified as practical contributors to stronger reasoning systems.
Supplementary Material
The supplementary material provides detailed parameter settings for the supervised fine-tuning and reinforcement learning stages.
- Supplementary Material: Table 1 lists the detailed training parameters used for SFT and RL.
1. Implementation Details
The implementation uses throughput-oriented SFT and RL configurations, unified evaluation, and answer validation to support efficient and consistent training assessment.
- SFT: SFT uses online stream packing with iterable data and a 61,440-token packing length to reduce padding and memory overhead.Models are trained until convergence because online packing prevents a predetermined epoch count.
- RL: RL uses a global batch size of 128, dynamic unpadded log-probability computation, fixed temperature 1.0, and training until reward saturation.Generation allows up to 28,696 new tokens with a 4,096-token prompt cap.
- Evaluation: SFT and RL share the same evaluation configuration, using a system prompt that elicits extractable reasoning traces and final answers.This supports consistent comparison across both training stages.
- Evaluation: Answers are validated first with a rule-based checker and then with an LLM judge when rule-based verification is insufficient.The two-stage process reduces evaluation cost while retaining fallback verification.
2. Additional Result and Analysis
Additional analyses show that scaling answer diversity improves benchmark performance, while RL training increases reflection-related language and a low format-reward weight performs best.
- Sampling Scaling Results: 49.2 average score, up from 46.7, when scaling the sampling strategy, demonstrating the effectiveness of increasing answer diversity.
- Rollout Analysis: As reward increases during RL training, the proportion of reflection-related words also rises in model outputs.Final rollouts frequently contain cues such as “let,” “wait,” and “think.”
- Reward λfmt Ablation: λfmt = 0.1 consistently yields the best performance among the tested format-reward weights and is adopted in the final configuration.The comparison tests λfmt values of 0.1, 0.3, 0.5, and 0.7 under the same GRPO setup.
3. Examples
The examples illustrate structured reasoning traces and qualitative robustness, while rollout analysis links RL progress with increased reflection language.
- Rollout Analysis: As RL training progresses, model outputs contain a higher proportion of reflection-related words.The final-step word cloud includes cues such as “let,” “wait,” and “think.”
- Sampling Scaling Results: Table 2 reports that scaling up answer generation improves results through greater answer diversity.
- Data examples: The reasoning examples decompose complex questions into structured steps and produce coherent, verifiable conclusions.
- Qualitative Results: Qualitative examples show accurate multimodal interpretation, coherent reasoning steps, and reliable answers across diverse scenarios.The examples are presented as evidence of generalization capability and practical utility.
4. Limitation and Future Work
The study is limited by its concentration on one model family and mainly image-based evaluation, while future work targets broader modalities and model configurations.
- Limitations: Evaluation focuses mainly on the Qwen2.5-VL-Instruct family and image-domain tasks, limiting evidence for broader modality and architecture generality.
- Limitations: The upper bound of performance under further SFT and RL scaling remains unidentified.
- Future Work: Future work aims to extend the methodology to video, audio, richer temporal streams, and broader model families.